[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:52:56 dist-test-slave-sms9 krb5kdc[7274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:52:56 dist-test-slave-sms9 krb5kdc[7274](info): set up 2 sockets
Jun 20 14:52:56 dist-test-slave-sms9 krb5kdc[7274](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260620 14:52:57.740161 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:52:57 dist-test-slave-sms9 krb5kdc[7274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:52:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:52:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:52:57.873373 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40819
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:40819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:52:57.982374 7287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:52:57.982739 7287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:52:57.982815 7287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:52:57.986856 7287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:52:57.986960 7287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:52:57.987000 7287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:52:57.987059 7287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:52:57.987098 7287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:52:57.992501 7287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:40819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:52:57.993602 7287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:52:57.994653 7287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:52:58.003836 7287 server_base.cc:1034] running on GCE node
W20260620 14:52:58.003783 7293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:52:58.003794 7295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:52:58.003783 7292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:52:58.004601 7287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:52:58.005767 7287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:52:58.006961 7287 hybrid_clock.cc:648] HybridClock initialized: now 1781967178006955 us; error 74 us; skew 500 ppm
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967178, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:52:58.009857 7287 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:52:58.010824 7287 webserver.cc:460] Webserver started at http://127.7.24.254:40631/ using document root <none> and password file <none>
I20260620 14:52:58.011411 7287 fs_manager.cc:359] Metadata directory not provided
I20260620 14:52:58.011502 7287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:52:58.011737 7287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:52:58.013792 7287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "a05874fc1d894ddca19f70d95bf4cece"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "35b30ff07ad32a51acf543f8ba963e04"
server_key_iv: "5705b4b4b9e9de8512de60e16a191732"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.014379 7287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "a05874fc1d894ddca19f70d95bf4cece"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "35b30ff07ad32a51acf543f8ba963e04"
server_key_iv: "5705b4b4b9e9de8512de60e16a191732"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.018126 7287 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:52:58.020754 7302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:52:58.022200 7287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:52:58.022349 7287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "a05874fc1d894ddca19f70d95bf4cece"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "35b30ff07ad32a51acf543f8ba963e04"
server_key_iv: "5705b4b4b9e9de8512de60e16a191732"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.022466 7287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:52:58.045995 7287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:52:58.049703 7287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:52:58.049923 7287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:52:58.057221 7287 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:40819
I20260620 14:52:58.057245 7354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:40819 every 8 connection(s)
I20260620 14:52:58.058194 7287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:52:58.060213 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 7287
I20260620 14:52:58.060400 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:52:58.061158 7355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:52:58.061924 7267 external_mini_cluster.cc:1428] Setting key 1f9925da50f9007b86df69d290bc142e
I20260620 14:52:58.067425 7355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece: Bootstrap starting.
I20260620 14:52:58.070232 7355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece: Neither blocks nor log segments found. Creating new log.
I20260620 14:52:58.071079 7355 log.cc:824] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece: Log is configured to *not* fsync() on all Append() calls
I20260620 14:52:58.073449 7355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece: No bootstrap required, opened a new log
I20260620 14:52:58.077869 7355 raft_consensus.cc:348] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05874fc1d894ddca19f70d95bf4cece" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40819 } }
I20260620 14:52:58.078092 7355 raft_consensus.cc:374] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:52:58.078125 7355 raft_consensus.cc:729] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a05874fc1d894ddca19f70d95bf4cece, State: Initialized, Role: FOLLOWER
I20260620 14:52:58.078660 7355 consensus_queue.cc:260] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05874fc1d894ddca19f70d95bf4cece" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40819 } }
I20260620 14:52:58.078816 7355 raft_consensus.cc:388] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:52:58.078907 7355 raft_consensus.cc:482] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:52:58.079023 7355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:52:58.080096 7355 raft_consensus.cc:504] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05874fc1d894ddca19f70d95bf4cece" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40819 } }
I20260620 14:52:58.080454 7355 leader_election.cc:302] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a05874fc1d894ddca19f70d95bf4cece; no voters:
I20260620 14:52:58.080783 7355 leader_election.cc:288] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:52:58.080864 7360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:52:58.081030 7360 raft_consensus.cc:686] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [term 1 LEADER]: Becoming Leader. State: Replica: a05874fc1d894ddca19f70d95bf4cece, State: Running, Role: LEADER
I20260620 14:52:58.081307 7360 consensus_queue.cc:237] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05874fc1d894ddca19f70d95bf4cece" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40819 } }
I20260620 14:52:58.082168 7355 sys_catalog.cc:564] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [sys.catalog]: configured and running, proceeding with master startup.
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:52:58.083278 7362 sys_catalog.cc:455] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a05874fc1d894ddca19f70d95bf4cece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05874fc1d894ddca19f70d95bf4cece" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40819 } } }
I20260620 14:52:58.083422 7362 sys_catalog.cc:458] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [sys.catalog]: This master's current role is: LEADER
I20260620 14:52:58.083797 7361 sys_catalog.cc:455] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [sys.catalog]: SysCatalogTable state changed. Reason: New leader a05874fc1d894ddca19f70d95bf4cece. Latest consensus state: current_term: 1 leader_uuid: "a05874fc1d894ddca19f70d95bf4cece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a05874fc1d894ddca19f70d95bf4cece" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40819 } } }
I20260620 14:52:58.084403 7361 sys_catalog.cc:458] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece [sys.catalog]: This master's current role is: LEADER
I20260620 14:52:58.084497 7369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:52:58.087728 7369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:52:58.090207 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:52:58.065308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45732 (local address 127.7.24.254:40819)
0620 14:52:58.065818 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0620 14:52:58.065841 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0620 14:52:58.067033 (+ 1192us) server_negotiation.cc:407] Connection header received
0620 14:52:58.071112 (+ 4079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:52:58.071129 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:52:58.071562 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:52:58.071929 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:52:58.074970 (+ 3041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.075999 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.077764 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.078363 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.078622 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:52:58.083945 (+ 5323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:52:58.083955 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:52:58.083975 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:52:58.084015 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:52:58.086212 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.086949 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.086955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.086959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.087079 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.087478 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.087488 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.087490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.087978 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:52:58.088213 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0620 14:52:58.088865 (+ 652us) server_negotiation.cc:299] Negotiation successful
0620 14:52:58.089148 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":134,"threads_started":1}
I20260620 14:52:58.094969 7369 catalog_manager.cc:1347] Generated new cluster ID: 3e1f2fdeb4584ec8958718b40768ecd7
I20260620 14:52:58.095067 7369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:52:58.106586 7369 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:52:58.107658 7369 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:52:58.115610 7369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a05874fc1d894ddca19f70d95bf4cece: Generated new TSK 0
I20260620 14:52:58.116381 7369 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:52:58.154366 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40819
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:52:58.303081 7381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:52:58.303337 7381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:52:58.303383 7381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:52:58.307350 7381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:52:58.307410 7381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:52:58.307471 7381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:52:58.312688 7381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40819
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:52:58.313696 7381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:52:58.314666 7381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:52:58.324097 7387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:52:58.324090 7386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:52:58.324219 7381 server_base.cc:1034] running on GCE node
W20260620 14:52:58.324399 7389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:52:58.324769 7381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:52:58.325383 7381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:52:58.326622 7381 hybrid_clock.cc:648] HybridClock initialized: now 1781967178326571 us; error 32 us; skew 500 ppm
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967178, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:52:58.329350 7381 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:52:58.330197 7381 webserver.cc:460] Webserver started at http://127.7.24.193:38121/ using document root <none> and password file <none>
I20260620 14:52:58.330780 7381 fs_manager.cc:359] Metadata directory not provided
I20260620 14:52:58.330842 7381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:52:58.331003 7381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:52:58.333010 7381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "156ecba9335341f1b9854b361ba2c7e1"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "890e27d81e67c15308cb54bf4ca741bc"
server_key_iv: "99c1d016a1aeb42327199e574b397df6"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.333534 7381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "156ecba9335341f1b9854b361ba2c7e1"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "890e27d81e67c15308cb54bf4ca741bc"
server_key_iv: "99c1d016a1aeb42327199e574b397df6"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.337184 7381 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 14:52:58.339643 7396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:52:58.340929 7381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:52:58.341025 7381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "156ecba9335341f1b9854b361ba2c7e1"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "890e27d81e67c15308cb54bf4ca741bc"
server_key_iv: "99c1d016a1aeb42327199e574b397df6"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.341109 7381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:52:58.350112 7381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:52:58.352800 7381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:52:58.352952 7381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:52:58.353430 7381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:52:58.354194 7381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:52:58.354252 7381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:52:58.354322 7381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:52:58.354362 7381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:52:58.363317 7381 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:34031
I20260620 14:52:58.363339 7509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:34031 every 8 connection(s)
I20260620 14:52:58.364179 7381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:52:58.372298 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 7381
I20260620 14:52:58.372435 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:52:58.372756 7267 external_mini_cluster.cc:1428] Setting key a3240df2344deb7922e17e95668d6b96
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967178, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:52:58.379081 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:52:58.366043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:52059 (local address 127.7.24.254:40819)
0620 14:52:58.366201 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 14:52:58.366204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:52:58.367049 (+ 845us) server_negotiation.cc:407] Connection header received
0620 14:52:58.368400 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:52:58.368403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:52:58.368464 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:52:58.368573 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:52:58.369751 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.370320 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.371530 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.371760 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.371878 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:52:58.374918 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:52:58.374922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:52:58.374925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:52:58.374953 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:52:58.376688 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.377347 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.377350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.377351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.377432 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.377861 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.377864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.377865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.378060 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:52:58.378143 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 14:52:58.378833 (+ 690us) server_negotiation.cc:299] Negotiation successful
0620 14:52:58.378954 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 14:52:58.380384 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:52:58.366365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40819 (local address 127.7.24.193:52059)
0620 14:52:58.366889 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0620 14:52:58.366932 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 14:52:58.368180 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:52:58.368751 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:52:58.368759 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:52:58.369127 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:52:58.369601 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:52:58.369614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.370453 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.370489 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:52:58.371382 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:52:58.371392 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.371919 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.371923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:52:58.372162 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:52:58.372805 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:52:58.372837 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:52:58.374752 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:52:58.376809 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:52:58.376819 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:52:58.376837 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:52:58.377219 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:52:58.377527 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:52:58.377533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:52:58.377535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:52:58.377729 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:52:58.378185 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:52:58.378195 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:52:58.378554 (+ 359us) client_negotiation.cc:769] Sending connection context
0620 14:52:58.378762 (+ 208us) client_negotiation.cc:240] Negotiation successful
0620 14:52:58.379153 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":116,"threads_started":1}
I20260620 14:52:58.381805 7510 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40819
I20260620 14:52:58.382098 7510 heartbeater.cc:464] Registering TS with master...
I20260620 14:52:58.382733 7510 heartbeater.cc:511] Master 127.7.24.254:40819 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:52:58.384356 7319 ts_manager.cc:194] Registered new tserver with Master: 156ecba9335341f1b9854b361ba2c7e1 (127.7.24.193:34031)
I20260620 14:52:58.385584 7319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:52059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:52:58.401511 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40819
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:52:58.512547 7515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:52:58.512817 7515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:52:58.512862 7515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:52:58.516983 7515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:52:58.517096 7515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:52:58.517220 7515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:52:58.522668 7515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40819
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:52:58.523708 7515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:52:58.524696 7515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:52:58.534284 7520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:52:58.534279 7521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:52:58.534365 7515 server_base.cc:1034] running on GCE node
W20260620 14:52:58.534284 7523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:52:58.535065 7515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:52:58.535790 7515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:52:58.537002 7515 hybrid_clock.cc:648] HybridClock initialized: now 1781967178536975 us; error 41 us; skew 500 ppm
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967178, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:52:58.539968 7515 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:52:58.540805 7515 webserver.cc:460] Webserver started at http://127.7.24.194:34575/ using document root <none> and password file <none>
I20260620 14:52:58.541360 7515 fs_manager.cc:359] Metadata directory not provided
I20260620 14:52:58.541476 7515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:52:58.541708 7515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:52:58.543795 7515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "9651523ccbae455f8efc45d3547920a3"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "57f8e312ea67960c0975a351f57b38a2"
server_key_iv: "756ba7b4aaf528974b7c535454408164"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.544725 7515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "9651523ccbae455f8efc45d3547920a3"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "57f8e312ea67960c0975a351f57b38a2"
server_key_iv: "756ba7b4aaf528974b7c535454408164"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.549047 7515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:52:58.551673 7530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:52:58.552999 7515 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:52:58.553159 7515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "9651523ccbae455f8efc45d3547920a3"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "57f8e312ea67960c0975a351f57b38a2"
server_key_iv: "756ba7b4aaf528974b7c535454408164"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.553283 7515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:52:58.578195 7515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:52:58.582008 7515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:52:58.582316 7515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:52:58.583130 7515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:52:58.584170 7515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:52:58.584245 7515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:52:58.584328 7515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:52:58.584403 7515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:52:58.594802 7515 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:36949
I20260620 14:52:58.594846 7643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:36949 every 8 connection(s)
I20260620 14:52:58.595753 7515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:52:58.598515 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 7515
I20260620 14:52:58.598615 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:52:58.598884 7267 external_mini_cluster.cc:1428] Setting key 7dd2c938c04dbc26235f897bdf511288
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967178, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:52:58.611433 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:52:58.597620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:33449 (local address 127.7.24.254:40819)
0620 14:52:58.597776 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 14:52:58.597779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:52:58.598714 (+ 935us) server_negotiation.cc:407] Connection header received
0620 14:52:58.600405 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:52:58.600408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:52:58.600461 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:52:58.600557 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:52:58.601726 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.602436 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.604081 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.604292 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.604380 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:52:58.607425 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:52:58.607429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:52:58.607435 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:52:58.607458 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:52:58.609175 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.609749 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.609754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.609757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.609852 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.610218 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.610224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.610226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.610432 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:52:58.610614 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0620 14:52:58.611129 (+ 515us) server_negotiation.cc:299] Negotiation successful
0620 14:52:58.611255 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
I20260620 14:52:58.612540 7646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:52:58.597989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40819 (local address 127.7.24.194:33449)
0620 14:52:58.598526 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0620 14:52:58.598601 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0620 14:52:58.600187 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:52:58.600707 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:52:58.600713 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:52:58.601114 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:52:58.601566 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:52:58.601578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.602600 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.602605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:52:58.603947 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:52:58.603961 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.604462 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.604470 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:52:58.604697 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:52:58.605415 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:52:58.605441 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:52:58.607242 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:52:58.609335 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:52:58.609341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:52:58.609356 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:52:58.609629 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:52:58.609957 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:52:58.609962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:52:58.609964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:52:58.610110 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:52:58.610610 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:52:58.610617 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:52:58.610876 (+ 259us) client_negotiation.cc:769] Sending connection context
0620 14:52:58.611083 (+ 207us) client_negotiation.cc:240] Negotiation successful
0620 14:52:58.611431 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":106,"threads_started":1}
I20260620 14:52:58.613914 7644 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40819
I20260620 14:52:58.614200 7644 heartbeater.cc:464] Registering TS with master...
I20260620 14:52:58.614830 7644 heartbeater.cc:511] Master 127.7.24.254:40819 requested a full tablet report, sending...
I20260620 14:52:58.616180 7319 ts_manager.cc:194] Registered new tserver with Master: 9651523ccbae455f8efc45d3547920a3 (127.7.24.194:36949)
I20260620 14:52:58.616815 7319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:33449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:52:58.629232 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40819
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:52:58.744505 7649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:52:58.744786 7649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:52:58.744838 7649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:52:58.748982 7649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:52:58.749048 7649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:52:58.749109 7649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:52:58.754554 7649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40819
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:52:58.755787 7649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:52:58.756865 7649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:52:58.766909 7657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:52:58.766957 7655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:52:58.767143 7649 server_base.cc:1034] running on GCE node
W20260620 14:52:58.766929 7654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:52:58.767741 7649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:52:58.768558 7649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:52:58.769829 7649 hybrid_clock.cc:648] HybridClock initialized: now 1781967178769749 us; error 96 us; skew 500 ppm
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967178, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:52:58.774111 7649 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:52:58.775332 7649 webserver.cc:460] Webserver started at http://127.7.24.195:34793/ using document root <none> and password file <none>
I20260620 14:52:58.776118 7649 fs_manager.cc:359] Metadata directory not provided
I20260620 14:52:58.776199 7649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:52:58.776378 7649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:52:58.778512 7649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "f7e313ac70cc41c4b34325303b323b3a"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "f661b174dab74aa65175345690063397"
server_key_iv: "61cd69dcf94a8421206abedaa44f3d09"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.779068 7649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "f7e313ac70cc41c4b34325303b323b3a"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "f661b174dab74aa65175345690063397"
server_key_iv: "61cd69dcf94a8421206abedaa44f3d09"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.783111 7649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 14:52:58.785809 7664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:52:58.787206 7649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:52:58.787352 7649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "f7e313ac70cc41c4b34325303b323b3a"
format_stamp: "Formatted at 2026-06-20 14:52:58 on dist-test-slave-sms9"
server_key: "f661b174dab74aa65175345690063397"
server_key_iv: "61cd69dcf94a8421206abedaa44f3d09"
server_key_version: "encryptionkey@0"
I20260620 14:52:58.787477 7649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:52:58.800114 7649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:52:58.803694 7649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:52:58.803915 7649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:52:58.804567 7649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:52:58.805478 7649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:52:58.805558 7649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:52:58.805639 7649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:52:58.805694 7649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:52:58.815132 7649 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:39727
I20260620 14:52:58.815300 7777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:39727 every 8 connection(s)
I20260620 14:52:58.816103 7649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:52:58.826035 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 7649
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967178, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:52:58.826165 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:52:58.826529 7267 external_mini_cluster.cc:1428] Setting key dc4b9b5ef09d608c7b5f1e7cba2c19bd
I20260620 14:52:58.830986 7357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:52:58.818058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:50775 (local address 127.7.24.254:40819)
0620 14:52:58.818200 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 14:52:58.818203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:52:58.819224 (+ 1021us) server_negotiation.cc:407] Connection header received
0620 14:52:58.820610 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:52:58.820614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:52:58.820671 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:52:58.820785 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:52:58.821986 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.822595 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.823747 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.823973 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.824087 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:52:58.826850 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:52:58.826854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:52:58.826857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:52:58.826883 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:52:58.828641 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.829388 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.829391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.829392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.829488 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.829843 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.829845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.829846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.830050 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:52:58.830136 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 14:52:58.830688 (+ 552us) server_negotiation.cc:299] Negotiation successful
0620 14:52:58.830795 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 14:52:58.831719 7780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:52:58.818593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40819 (local address 127.7.24.195:50775)
0620 14:52:58.819071 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0620 14:52:58.819114 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 14:52:58.820392 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:52:58.820957 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:52:58.820964 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:52:58.821327 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:52:58.821840 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:52:58.821853 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.822719 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.822723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:52:58.823616 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:52:58.823625 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.824125 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.824129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:52:58.824296 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:52:58.824917 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:52:58.824934 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:52:58.826683 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:52:58.828776 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:52:58.828780 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:52:58.828792 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:52:58.829223 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:52:58.829606 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:52:58.829608 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:52:58.829610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:52:58.829750 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:52:58.830145 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:52:58.830150 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:52:58.830394 (+ 244us) client_negotiation.cc:769] Sending connection context
0620 14:52:58.830622 (+ 228us) client_negotiation.cc:240] Negotiation successful
0620 14:52:58.830905 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":126,"threads_started":1}
I20260620 14:52:58.832841 7778 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40819
I20260620 14:52:58.833142 7778 heartbeater.cc:464] Registering TS with master...
I20260620 14:52:58.833686 7778 heartbeater.cc:511] Master 127.7.24.254:40819 requested a full tablet report, sending...
I20260620 14:52:58.834830 7319 ts_manager.cc:194] Registered new tserver with Master: f7e313ac70cc41c4b34325303b323b3a (127.7.24.195:39727)
I20260620 14:52:58.835543 7319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:50775
I20260620 14:52:58.841310 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967178, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967178, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:52:58.870623 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:52:58.859637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48094 (local address 127.7.24.193:34031)
0620 14:52:58.859937 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0620 14:52:58.859954 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 14:52:58.860110 (+ 156us) server_negotiation.cc:407] Connection header received
0620 14:52:58.860315 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:52:58.860322 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:52:58.860557 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:52:58.860728 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:52:58.861509 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.862369 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.863370 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.863902 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.864059 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:52:58.867039 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:52:58.867050 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:52:58.867065 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:52:58.867092 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:52:58.868745 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.869194 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.869199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.869200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.869282 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.869560 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.869563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.869564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.869822 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:52:58.870065 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0620 14:52:58.870235 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 14:52:58.870367 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":71,"threads_started":1}
W20260620 14:52:58.871656 7423 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48094
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:52:58.899080 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:52:58.889901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.7.24.193:34031)
0620 14:52:58.890063 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 14:52:58.890067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:52:58.890082 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:52:58.890229 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:52:58.890233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:52:58.890287 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:52:58.890361 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:52:58.891271 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.891845 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.892905 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:52:58.893144 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:52:58.893228 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:52:58.895886 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:52:58.895890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:52:58.895893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:52:58.895918 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:52:58.897252 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.897772 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.897775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.897776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.897854 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:52:58.898127 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:52:58.898129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:52:58.898130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:52:58.898292 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:52:58.898390 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 14:52:58.898732 (+ 342us) server_negotiation.cc:299] Negotiation successful
0620 14:52:58.898840 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 14:52:58.900148 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 7381
I20260620 14:52:58.905915 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 7515
I20260620 14:52:58.911652 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 7649
I20260620 14:52:58.917323 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 7287
2026-06-20T14:52:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2252 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7792](info): set up 2 sockets
Jun 20 14:52:58 dist-test-slave-sms9 krb5kdc[7792](info): commencing operation
krb5kdc: starting...
W20260620 14:52:59.986238 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:00 dist-test-slave-sms9 krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:00.109754 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:34743
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:39615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:34743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:00.225045 7805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:00.225350 7805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:00.225394 7805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:00.229640 7805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:00.229720 7805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:00.229740 7805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:00.229758 7805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:00.229774 7805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:00.236172 7805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:39615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:34743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:34743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:00.237414 7805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:00.238616 7805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:00.247830 7813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:00.247830 7810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:00.247989 7805 server_base.cc:1034] running on GCE node
W20260620 14:53:00.247917 7811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:00.248729 7805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:00.250067 7805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:00.251302 7805 hybrid_clock.cc:648] HybridClock initialized: now 1781967180251254 us; error 71 us; skew 500 ppm
Jun 20 14:53:00 dist-test-slave-sms9 krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967180, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:00.258889 7805 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:00.259980 7805 webserver.cc:460] Webserver started at http://127.7.24.254:38983/ using document root <none> and password file <none>
I20260620 14:53:00.260586 7805 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:00.260679 7805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:00.260927 7805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:00.263144 7805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "2510a599388e4d92b592857b7e4eeb01"
format_stamp: "Formatted at 2026-06-20 14:53:00 on dist-test-slave-sms9"
server_key: "e84bf93b78a6544ec5a7b882a59cbf69"
server_key_iv: "1bc79d29710aac9da398f2b7f80f17fb"
server_key_version: "encryptionkey@0"
I20260620 14:53:00.263747 7805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "2510a599388e4d92b592857b7e4eeb01"
format_stamp: "Formatted at 2026-06-20 14:53:00 on dist-test-slave-sms9"
server_key: "e84bf93b78a6544ec5a7b882a59cbf69"
server_key_iv: "1bc79d29710aac9da398f2b7f80f17fb"
server_key_version: "encryptionkey@0"
I20260620 14:53:00.267697 7805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260620 14:53:00.270429 7820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:00.271934 7805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 14:53:00.272104 7805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "2510a599388e4d92b592857b7e4eeb01"
format_stamp: "Formatted at 2026-06-20 14:53:00 on dist-test-slave-sms9"
server_key: "e84bf93b78a6544ec5a7b882a59cbf69"
server_key_iv: "1bc79d29710aac9da398f2b7f80f17fb"
server_key_version: "encryptionkey@0"
I20260620 14:53:00.272243 7805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:00.287366 7805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:00.290978 7805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:00.291208 7805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:00.299425 7805 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:34743
I20260620 14:53:00.299446 7872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:34743 every 8 connection(s)
I20260620 14:53:00.300488 7805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:00.303505 7873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:00.307152 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 7805
I20260620 14:53:00.307255 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:00.307533 7267 external_mini_cluster.cc:1428] Setting key c261d311528c7e64ef8d92a88fb69543
I20260620 14:53:00.310112 7873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01: Bootstrap starting.
I20260620 14:53:00.313038 7873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:00.313954 7873 log.cc:824] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:00.316567 7873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01: No bootstrap required, opened a new log
Jun 20 14:53:00 dist-test-slave-sms9 krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:00.321085 7873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2510a599388e4d92b592857b7e4eeb01" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34743 } }
I20260620 14:53:00.321410 7873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:00.321483 7873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2510a599388e4d92b592857b7e4eeb01, State: Initialized, Role: FOLLOWER
I20260620 14:53:00.321997 7873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2510a599388e4d92b592857b7e4eeb01" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34743 } }
I20260620 14:53:00.322160 7873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:00.322237 7873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:00.322348 7873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:00.323742 7873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2510a599388e4d92b592857b7e4eeb01" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34743 } }
I20260620 14:53:00.324144 7873 leader_election.cc:302] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2510a599388e4d92b592857b7e4eeb01; no voters:
I20260620 14:53:00.324515 7873 leader_election.cc:288] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:00.324659 7878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:00.324910 7878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [term 1 LEADER]: Becoming Leader. State: Replica: 2510a599388e4d92b592857b7e4eeb01, State: Running, Role: LEADER
I20260620 14:53:00.325256 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:00.309135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46794 (local address 127.7.24.254:34743)
0620 14:53:00.309618 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:00.309638 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:00.309683 (+ 45us) server_negotiation.cc:407] Connection header received
0620 14:53:00.310570 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:00.310596 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:00.311027 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:00.311431 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:00.312164 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:00.313675 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:00.314907 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:00.315605 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:00.315807 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:00.318675 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:00.318685 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:00.318707 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:00.318744 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:00.321615 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:00.322218 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:00.322224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:00.322231 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:00.322401 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:00.322797 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:00.322800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:00.322802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:00.323377 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:00.323594 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:00.323896 (+ 302us) server_negotiation.cc:299] Negotiation successful
0620 14:53:00.324219 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":146,"threads_started":1}
I20260620 14:53:00.325256 7878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2510a599388e4d92b592857b7e4eeb01" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34743 } }
I20260620 14:53:00.325980 7873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:00.327193 7879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2510a599388e4d92b592857b7e4eeb01. Latest consensus state: current_term: 1 leader_uuid: "2510a599388e4d92b592857b7e4eeb01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2510a599388e4d92b592857b7e4eeb01" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34743 } } }
I20260620 14:53:00.327381 7879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:00.327724 7880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2510a599388e4d92b592857b7e4eeb01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2510a599388e4d92b592857b7e4eeb01" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34743 } } }
I20260620 14:53:00.327814 7880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01 [sys.catalog]: This master's current role is: LEADER
W20260620 14:53:00.331976 7893 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:53:00.332068 7893 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:53:00.332166 7887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:00.333107 7887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:00.339408 7887 catalog_manager.cc:1347] Generated new cluster ID: f06c4eef79d04ca0884c5092f9a1e0f3
I20260620 14:53:00.339483 7887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:00.367093 7887 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:00.368072 7887 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:00.382678 7887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2510a599388e4d92b592857b7e4eeb01: Generated new TSK 0
I20260620 14:53:00.383599 7887 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:53:00.417585 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34743
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:39615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:00.550825 7899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:00.551065 7899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:00.551108 7899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:00.555089 7899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:00.555141 7899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:00.555198 7899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:00.560338 7899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:39615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34743
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:00.561261 7899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:00.562187 7899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:00.571897 7904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:00.572019 7905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:00.572101 7899 server_base.cc:1034] running on GCE node
W20260620 14:53:00.572118 7907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:00.572598 7899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:00.573293 7899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:00.574520 7899 hybrid_clock.cc:648] HybridClock initialized: now 1781967180574500 us; error 28 us; skew 500 ppm
Jun 20 14:53:00 dist-test-slave-sms9 krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967180, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:00.577478 7899 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:53:00.578338 7899 webserver.cc:460] Webserver started at http://127.7.24.193:45251/ using document root <none> and password file <none>
I20260620 14:53:00.578995 7899 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:00.579094 7899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:00.579329 7899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:00.581431 7899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "54c392e12ff445ba942e0fbd33d9c5ce"
format_stamp: "Formatted at 2026-06-20 14:53:00 on dist-test-slave-sms9"
server_key: "9d3fbc64552823e5d811b1260add5352"
server_key_iv: "ffa41b10090f489dd6ea5c11c0df89d8"
server_key_version: "encryptionkey@0"
I20260620 14:53:00.582013 7899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "54c392e12ff445ba942e0fbd33d9c5ce"
format_stamp: "Formatted at 2026-06-20 14:53:00 on dist-test-slave-sms9"
server_key: "9d3fbc64552823e5d811b1260add5352"
server_key_iv: "ffa41b10090f489dd6ea5c11c0df89d8"
server_key_version: "encryptionkey@0"
I20260620 14:53:00.585714 7899 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 14:53:00.588217 7914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:00.589453 7899 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:00.589587 7899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "54c392e12ff445ba942e0fbd33d9c5ce"
format_stamp: "Formatted at 2026-06-20 14:53:00 on dist-test-slave-sms9"
server_key: "9d3fbc64552823e5d811b1260add5352"
server_key_iv: "ffa41b10090f489dd6ea5c11c0df89d8"
server_key_version: "encryptionkey@0"
I20260620 14:53:00.589695 7899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:00.607455 7899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:00.610743 7899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:00.610957 7899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:00.611558 7899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:00.612376 7899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:00.612449 7899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:00.612522 7899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:00.612567 7899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:00.621642 7899 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:34635
I20260620 14:53:00.621675 8027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:34635 every 8 connection(s)
I20260620 14:53:00.622526 7899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:00.625694 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 7899
I20260620 14:53:00.625813 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:00.626145 7267 external_mini_cluster.cc:1428] Setting key b715964e7f0209cff23b9b0c20f77978
Jun 20 14:53:00 dist-test-slave-sms9 krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967180, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:53:00.637622 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:00.624365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:46205 (local address 127.7.24.254:34743)
0620 14:53:00.624529 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:00.624534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:00.625266 (+ 732us) server_negotiation.cc:407] Connection header received
0620 14:53:00.627120 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:00.627123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:00.627179 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:00.627253 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:00.628878 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:00.629596 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:00.630877 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:00.631108 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:00.631224 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:00.633687 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:00.633691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:00.633694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:00.633725 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:00.635306 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:00.635909 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:00.635915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:00.635917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:00.636006 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:00.636359 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:00.636362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:00.636364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:00.636644 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:00.636753 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:00.637249 (+ 496us) server_negotiation.cc:299] Negotiation successful
0620 14:53:00.637378 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":65}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
I20260620 14:53:00.638612 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:00.624690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:34743 (local address 127.7.24.193:46205)
0620 14:53:00.625121 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:00.625162 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:00.626865 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:00.627469 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:00.627481 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:00.627982 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:00.628677 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:00.628698 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:00.629730 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:00.629732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:00.630746 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:00.630755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:00.631259 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:00.631263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:00.631440 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:00.632104 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:00.632120 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:00.633523 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:00.635429 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:00.635435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:00.635448 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:00.635748 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:00.636101 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:00.636105 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:00.636107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:00.636243 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:00.636752 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:00.636759 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:00.637006 (+ 247us) client_negotiation.cc:769] Sending connection context
0620 14:53:00.637196 (+ 190us) client_negotiation.cc:240] Negotiation successful
0620 14:53:00.637545 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":110,"threads_started":1}
I20260620 14:53:00.641069 8028 heartbeater.cc:347] Connected to a master server at 127.7.24.254:34743
I20260620 14:53:00.641345 8028 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:00.641842 8028 heartbeater.cc:511] Master 127.7.24.254:34743 requested a full tablet report, sending...
I20260620 14:53:00.643330 7837 ts_manager.cc:194] Registered new tserver with Master: 54c392e12ff445ba942e0fbd33d9c5ce (127.7.24.193:34635)
I20260620 14:53:00.644428 7837 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:46205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:53:00.653553 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34743
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:39615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:00.764945 8033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:00.765229 8033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:00.765275 8033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:00.769595 8033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:00.769663 8033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:00.769780 8033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:00.775301 8033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:39615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34743
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:00.776460 8033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:00.777493 8033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:00.787245 8041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:00.787395 8033 server_base.cc:1034] running on GCE node
W20260620 14:53:00.787245 8039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:00.787585 8038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:00.788009 8033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:00.788700 8033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:00.789916 8033 hybrid_clock.cc:648] HybridClock initialized: now 1781967180789893 us; error 48 us; skew 500 ppm
Jun 20 14:53:00 dist-test-slave-sms9 krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967180, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:00.792728 8033 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:53:00.793726 8033 webserver.cc:460] Webserver started at http://127.7.24.194:38549/ using document root <none> and password file <none>
I20260620 14:53:00.794302 8033 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:00.794399 8033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:00.794656 8033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:00.796875 8033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "722d652632a14df2bbfb2c57aaed2786"
format_stamp: "Formatted at 2026-06-20 14:53:00 on dist-test-slave-sms9"
server_key: "90ba6a698a3a973b3023ba470ccaffe8"
server_key_iv: "056aa4fdb8614740302e9d31bb7f970f"
server_key_version: "encryptionkey@0"
I20260620 14:53:00.797564 8033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "722d652632a14df2bbfb2c57aaed2786"
format_stamp: "Formatted at 2026-06-20 14:53:00 on dist-test-slave-sms9"
server_key: "90ba6a698a3a973b3023ba470ccaffe8"
server_key_iv: "056aa4fdb8614740302e9d31bb7f970f"
server_key_version: "encryptionkey@0"
I20260620 14:53:00.801448 8033 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:53:00.804024 8048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:00.805263 8033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:00.805403 8033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "722d652632a14df2bbfb2c57aaed2786"
format_stamp: "Formatted at 2026-06-20 14:53:00 on dist-test-slave-sms9"
server_key: "90ba6a698a3a973b3023ba470ccaffe8"
server_key_iv: "056aa4fdb8614740302e9d31bb7f970f"
server_key_version: "encryptionkey@0"
I20260620 14:53:00.805523 8033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:00.829425 8033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:00.832532 8033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:00.832739 8033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:00.833351 8033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:00.834221 8033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:00.834291 8033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:00.834363 8033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:00.834416 8033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:00.843930 8033 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:38613
I20260620 14:53:00.843950 8161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:38613 every 8 connection(s)
I20260620 14:53:00.844861 8033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:00.850214 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 8033
I20260620 14:53:00.850348 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:00.850675 7267 external_mini_cluster.cc:1428] Setting key ba904043a010bd111a09906d26e0d5c2
Jun 20 14:53:00 dist-test-slave-sms9 krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967180, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:00.861027 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:00.846725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:33369 (local address 127.7.24.254:34743)
0620 14:53:00.846920 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:00.846924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:00.847761 (+ 837us) server_negotiation.cc:407] Connection header received
0620 14:53:00.849104 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:00.849106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:00.849153 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:00.849230 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:00.850712 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:00.851440 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:00.852817 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:00.853047 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:00.853132 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:00.856386 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:00.856401 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:00.856408 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:00.856447 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:00.858383 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:00.859071 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:00.859076 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:00.859080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:00.859181 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:00.859580 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:00.859585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:00.859588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:00.859867 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:00.859979 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:00.860679 (+ 700us) server_negotiation.cc:299] Negotiation successful
0620 14:53:00.860842 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:53:00.861950 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:00.847157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:34743 (local address 127.7.24.194:33369)
0620 14:53:00.847602 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:00.847646 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:00.848899 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:00.849402 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:00.849409 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:00.849765 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:00.850532 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:00.850553 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:00.851562 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:00.851565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:00.852684 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:00.852695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:00.853209 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:00.853216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:00.853472 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:00.854259 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:00.854276 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:00.856180 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:00.858588 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:00.858594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:00.858611 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:00.858938 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:00.859298 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:00.859301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:00.859303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:00.859465 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:00.859996 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:00.860003 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:00.860281 (+ 278us) client_negotiation.cc:769] Sending connection context
0620 14:53:00.860595 (+ 314us) client_negotiation.cc:240] Negotiation successful
0620 14:53:00.860982 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":106,"threads_started":1}
I20260620 14:53:00.863355 8162 heartbeater.cc:347] Connected to a master server at 127.7.24.254:34743
I20260620 14:53:00.863607 8162 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
I20260620 14:53:00.864337 8162 heartbeater.cc:511] Master 127.7.24.254:34743 requested a full tablet report, sending...
I20260620 14:53:00.865538 7837 ts_manager.cc:194] Registered new tserver with Master: 722d652632a14df2bbfb2c57aaed2786 (127.7.24.194:38613)
I20260620 14:53:00.866160 7837 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:33369
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:53:00.882786 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34743
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:39615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:00.994501 8167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:00.994810 8167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:00.994889 8167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:00.999007 8167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:00.999099 8167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:00.999197 8167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:01.004460 8167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:39615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34743
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:01.005549 8167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:01.006614 8167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:01.016446 8173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:01.016450 8172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:01.016470 8175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:01.016736 8167 server_base.cc:1034] running on GCE node
I20260620 14:53:01.017098 8167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:01.017710 8167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:01.018890 8167 hybrid_clock.cc:648] HybridClock initialized: now 1781967181018872 us; error 37 us; skew 500 ppm
Jun 20 14:53:01 dist-test-slave-sms9 krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967181, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:01.021796 8167 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:53:01.022797 8167 webserver.cc:460] Webserver started at http://127.7.24.195:44119/ using document root <none> and password file <none>
I20260620 14:53:01.023324 8167 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:01.023391 8167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:01.023559 8167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:01.025533 8167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "b7f0fb65566d4d74bb8068ef48bded0a"
format_stamp: "Formatted at 2026-06-20 14:53:01 on dist-test-slave-sms9"
server_key: "75f2880584f8c1495d17c6ae55f9172d"
server_key_iv: "f8427551c6a3e85c90dbf463cdbd0578"
server_key_version: "encryptionkey@0"
I20260620 14:53:01.026081 8167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "b7f0fb65566d4d74bb8068ef48bded0a"
format_stamp: "Formatted at 2026-06-20 14:53:01 on dist-test-slave-sms9"
server_key: "75f2880584f8c1495d17c6ae55f9172d"
server_key_iv: "f8427551c6a3e85c90dbf463cdbd0578"
server_key_version: "encryptionkey@0"
I20260620 14:53:01.030071 8167 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:53:01.032590 8182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:01.033824 8167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:01.033927 8167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "b7f0fb65566d4d74bb8068ef48bded0a"
format_stamp: "Formatted at 2026-06-20 14:53:01 on dist-test-slave-sms9"
server_key: "75f2880584f8c1495d17c6ae55f9172d"
server_key_iv: "f8427551c6a3e85c90dbf463cdbd0578"
server_key_version: "encryptionkey@0"
I20260620 14:53:01.034022 8167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:01.054669 8167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:01.058054 8167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:01.058218 8167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:01.058843 8167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:01.059716 8167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:01.059764 8167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:01.059806 8167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:01.059855 8167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:01.069664 8167 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:35423
I20260620 14:53:01.069680 8295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:35423 every 8 connection(s)
I20260620 14:53:01.070665 8167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:01.079455 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 8167
I20260620 14:53:01.079555 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:01.079831 7267 external_mini_cluster.cc:1428] Setting key 5fd8a22faed2eb63773dec847fd33d07
Jun 20 14:53:01 dist-test-slave-sms9 krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967181, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:01.084801 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.072456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:36177 (local address 127.7.24.254:34743)
0620 14:53:01.072616 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:01.072620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:01.073532 (+ 912us) server_negotiation.cc:407] Connection header received
0620 14:53:01.074978 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:01.074981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:01.075040 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:01.075151 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:01.076387 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.076924 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.078088 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.078313 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.078435 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.081223 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:01.081226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:01.081230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:01.081260 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:01.082990 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:01.083483 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:01.083485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:01.083487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:01.083561 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:01.083843 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:01.083845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:01.083846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:01.084014 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:01.084088 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:01.084577 (+ 489us) server_negotiation.cc:299] Negotiation successful
0620 14:53:01.084680 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 14:53:01.085703 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.072846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:34743 (local address 127.7.24.195:36177)
0620 14:53:01.073280 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:01.073323 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:01.074750 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:01.075337 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:01.075344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:01.075728 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:01.076237 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.076249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.077054 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.077058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.077955 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.077965 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.078498 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.078502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.078681 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.079255 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:01.079271 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:01.081012 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:01.083096 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:01.083101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:01.083115 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:01.083394 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:01.083637 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:01.083639 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:01.083640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:01.083768 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:01.084100 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:01.084106 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:01.084363 (+ 257us) client_negotiation.cc:769] Sending connection context
0620 14:53:01.084524 (+ 161us) client_negotiation.cc:240] Negotiation successful
0620 14:53:01.084783 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":103,"threads_started":1}
I20260620 14:53:01.086959 8296 heartbeater.cc:347] Connected to a master server at 127.7.24.254:34743
I20260620 14:53:01.087165 8296 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:01.087607 8296 heartbeater.cc:511] Master 127.7.24.254:34743 requested a full tablet report, sending...
I20260620 14:53:01.088617 7837 ts_manager.cc:194] Registered new tserver with Master: b7f0fb65566d4d74bb8068ef48bded0a (127.7.24.195:35423)
I20260620 14:53:01.089217 7837 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:36177
I20260620 14:53:01.094211 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:53:01.105507 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.097951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46798 (local address 127.7.24.254:34743)
0620 14:53:01.098087 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:01.098091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:01.098296 (+ 205us) server_negotiation.cc:407] Connection header received
0620 14:53:01.098369 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:01.098371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:01.098416 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:01.098767 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:01.099511 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.100027 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.100845 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.101067 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.101156 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.102707 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:01.102710 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:01.102712 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:01.102737 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:01.104014 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:01.104425 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:01.104428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:01.104429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:01.104507 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:01.104752 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:01.104754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:01.104755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:01.104977 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:01.105085 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:01.105283 (+ 198us) server_negotiation.cc:299] Negotiation successful
0620 14:53:01.105386 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 14:53:01.110255 7837 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:01.113270 7837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:01.127568 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.121613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38613 (local address 127.0.0.1:39636)
0620 14:53:01.122223 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:01.122236 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:01.122317 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:01.123552 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:01.123557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:01.123586 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:01.123761 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.123768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.124866 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.124872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.125835 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.125844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.126998 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.127002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.127111 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.127148 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 14:53:01.127257 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 14:53:01.127378 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":97,"spinlock_wait_cycles":5504,"thread_start_us":91,"threads_started":1}
I20260620 14:53:01.127724 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.121556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43346 (local address 127.7.24.195:35423)
0620 14:53:01.122079 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:01.122092 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:01.122117 (+ 25us) server_negotiation.cc:407] Connection header received
0620 14:53:01.122194 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:01.122200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:01.122420 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:01.122597 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:01.123044 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.124180 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.125908 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.126804 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.126979 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.127134 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:01.127473 (+ 339us) server_negotiation.cc:299] Negotiation successful
0620 14:53:01.127569 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":160,"threads_started":1}
I20260620 14:53:01.127736 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.121415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:35423 (local address 127.0.0.1:43346)
0620 14:53:01.121756 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:01.121802 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:01.121918 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:01.122752 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:01.122757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:01.122784 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:01.122888 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.122892 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.124321 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.124325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.125699 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.125709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.127191 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.127194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.127319 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.127341 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 14:53:01.127428 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 14:53:01.127512 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":121,"threads_started":1}
I20260620 14:53:01.128070 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.122331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.7.24.194:38613)
0620 14:53:01.123126 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:01.123137 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:01.123158 (+ 21us) server_negotiation.cc:407] Connection header received
0620 14:53:01.123229 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:01.123236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:01.123410 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:01.123652 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:01.123896 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.124735 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.126049 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.126828 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.127001 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.127181 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:01.127773 (+ 592us) server_negotiation.cc:299] Negotiation successful
0620 14:53:01.127925 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":237,"threads_started":1}
I20260620 14:53:01.128510 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.122596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44378 (local address 127.7.24.193:34635)
0620 14:53:01.122851 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:01.122865 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:01.122887 (+ 22us) server_negotiation.cc:407] Connection header received
0620 14:53:01.122965 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:01.122972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:01.123146 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:01.123290 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:01.123848 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.124762 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.126293 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.127118 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.127215 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.127339 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:01.127596 (+ 257us) server_negotiation.cc:299] Negotiation successful
0620 14:53:01.128368 (+ 772us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":77,"threads_started":1}
I20260620 14:53:01.128871 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.122134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:34635 (local address 127.0.0.1:44378)
0620 14:53:01.122535 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:01.122556 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:01.122628 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:01.123562 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:01.123564 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:01.123587 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:01.123745 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.123751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.124928 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.124931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.126132 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.126141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.127257 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.127260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.127341 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.127360 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 14:53:01.127424 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 14:53:01.127495 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":95,"spinlock_wait_cycles":4096,"thread_start_us":88,"threads_started":1}
I20260620 14:53:01.130831 8096 tablet_service.cc:1467] Processing CreateTablet for tablet 449fb0b7394e4a59bd1e655c5710b58a (DEFAULT_TABLE table=test-table [id=ba313a66702c460c98ec8141cddfe151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:01.130842 8230 tablet_service.cc:1467] Processing CreateTablet for tablet 449fb0b7394e4a59bd1e655c5710b58a (DEFAULT_TABLE table=test-table [id=ba313a66702c460c98ec8141cddfe151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:01.131253 7962 tablet_service.cc:1467] Processing CreateTablet for tablet 449fb0b7394e4a59bd1e655c5710b58a (DEFAULT_TABLE table=test-table [id=ba313a66702c460c98ec8141cddfe151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:01.131855 8096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 449fb0b7394e4a59bd1e655c5710b58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:01.131860 8230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 449fb0b7394e4a59bd1e655c5710b58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:01.132119 7962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 449fb0b7394e4a59bd1e655c5710b58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:01.139182 8313 tablet_bootstrap.cc:492] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786: Bootstrap starting.
I20260620 14:53:01.139693 8314 tablet_bootstrap.cc:492] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce: Bootstrap starting.
I20260620 14:53:01.141364 8314 tablet_bootstrap.cc:654] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:01.141779 8313 tablet_bootstrap.cc:654] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:01.142206 8314 log.cc:824] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:01.142532 8313 log.cc:824] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:01.142822 8315 tablet_bootstrap.cc:492] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a: Bootstrap starting.
I20260620 14:53:01.144317 8314 tablet_bootstrap.cc:492] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce: No bootstrap required, opened a new log
I20260620 14:53:01.144435 8313 tablet_bootstrap.cc:492] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786: No bootstrap required, opened a new log
I20260620 14:53:01.144527 8314 ts_tablet_manager.cc:1397] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260620 14:53:01.144598 8313 ts_tablet_manager.cc:1397] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:53:01.145424 8315 tablet_bootstrap.cc:654] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:01.146204 8315 log.cc:824] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:01.148137 8314 raft_consensus.cc:348] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f0fb65566d4d74bb8068ef48bded0a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 35423 } } peers { permanent_uuid: "54c392e12ff445ba942e0fbd33d9c5ce" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34635 } } peers { permanent_uuid: "722d652632a14df2bbfb2c57aaed2786" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38613 } }
I20260620 14:53:01.148377 8314 raft_consensus.cc:374] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:01.148399 8315 tablet_bootstrap.cc:492] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a: No bootstrap required, opened a new log
I20260620 14:53:01.148437 8314 raft_consensus.cc:729] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c392e12ff445ba942e0fbd33d9c5ce, State: Initialized, Role: FOLLOWER
I20260620 14:53:01.148579 8315 ts_tablet_manager.cc:1397] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:53:01.148522 8313 raft_consensus.cc:348] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f0fb65566d4d74bb8068ef48bded0a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 35423 } } peers { permanent_uuid: "54c392e12ff445ba942e0fbd33d9c5ce" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34635 } } peers { permanent_uuid: "722d652632a14df2bbfb2c57aaed2786" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38613 } }
I20260620 14:53:01.148780 8313 raft_consensus.cc:374] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:01.148844 8313 raft_consensus.cc:729] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 722d652632a14df2bbfb2c57aaed2786, State: Initialized, Role: FOLLOWER
I20260620 14:53:01.148983 8314 consensus_queue.cc:260] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f0fb65566d4d74bb8068ef48bded0a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 35423 } } peers { permanent_uuid: "54c392e12ff445ba942e0fbd33d9c5ce" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34635 } } peers { permanent_uuid: "722d652632a14df2bbfb2c57aaed2786" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38613 } }
I20260620 14:53:01.149230 8313 consensus_queue.cc:260] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f0fb65566d4d74bb8068ef48bded0a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 35423 } } peers { permanent_uuid: "54c392e12ff445ba942e0fbd33d9c5ce" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34635 } } peers { permanent_uuid: "722d652632a14df2bbfb2c57aaed2786" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38613 } }
I20260620 14:53:01.149706 8028 heartbeater.cc:503] Master 127.7.24.254:34743 was elected leader, sending a full tablet report...
I20260620 14:53:01.149981 8314 ts_tablet_manager.cc:1428] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 14:53:01.150106 8313 ts_tablet_manager.cc:1428] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 14:53:01.150431 8162 heartbeater.cc:503] Master 127.7.24.254:34743 was elected leader, sending a full tablet report...
I20260620 14:53:01.153664 8315 raft_consensus.cc:348] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f0fb65566d4d74bb8068ef48bded0a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 35423 } } peers { permanent_uuid: "54c392e12ff445ba942e0fbd33d9c5ce" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34635 } } peers { permanent_uuid: "722d652632a14df2bbfb2c57aaed2786" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38613 } }
I20260620 14:53:01.153950 8315 raft_consensus.cc:374] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:01.154023 8315 raft_consensus.cc:729] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f0fb65566d4d74bb8068ef48bded0a, State: Initialized, Role: FOLLOWER
I20260620 14:53:01.154552 8315 consensus_queue.cc:260] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f0fb65566d4d74bb8068ef48bded0a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 35423 } } peers { permanent_uuid: "54c392e12ff445ba942e0fbd33d9c5ce" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34635 } } peers { permanent_uuid: "722d652632a14df2bbfb2c57aaed2786" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38613 } }
I20260620 14:53:01.155218 8296 heartbeater.cc:503] Master 127.7.24.254:34743 was elected leader, sending a full tablet report...
I20260620 14:53:01.155545 8315 ts_tablet_manager.cc:1428] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260620 14:53:01.245937 8320 raft_consensus.cc:482] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:01.246132 8320 raft_consensus.cc:504] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f0fb65566d4d74bb8068ef48bded0a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 35423 } } peers { permanent_uuid: "54c392e12ff445ba942e0fbd33d9c5ce" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34635 } } peers { permanent_uuid: "722d652632a14df2bbfb2c57aaed2786" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38613 } }
I20260620 14:53:01.247342 8320 leader_election.cc:288] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7f0fb65566d4d74bb8068ef48bded0a (127.7.24.195:35423), 54c392e12ff445ba942e0fbd33d9c5ce (127.7.24.193:34635)
I20260620 14:53:01.251117 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.247592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:35423 (local address 127.7.24.194:47919)
0620 14:53:01.247724 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:01.247742 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:01.247844 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:01.248128 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:01.248131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:01.248151 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:01.248286 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.248293 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.249106 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.249110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.249938 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.249948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.250683 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.250686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.250797 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.250816 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 14:53:01.250904 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 14:53:01.250978 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 14:53:01.251117 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.247681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:47919 (local address 127.7.24.195:35423)
0620 14:53:01.247822 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:01.247825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:01.247866 (+ 41us) server_negotiation.cc:407] Connection header received
0620 14:53:01.247934 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:01.247940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:01.247995 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:01.248108 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:01.248419 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.248980 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.250075 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.250559 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.250685 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.250739 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:01.250931 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 14:53:01.251003 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:53:01.251577 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.248056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:41715 (local address 127.7.24.193:34635)
0620 14:53:01.248245 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:01.248250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:01.248418 (+ 168us) server_negotiation.cc:407] Connection header received
0620 14:53:01.248578 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:01.248581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:01.248631 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:01.248711 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:01.249000 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.249492 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.250573 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.250966 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.251099 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.251146 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:01.251356 (+ 210us) server_negotiation.cc:299] Negotiation successful
0620 14:53:01.251429 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260620 14:53:01.251935 8322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.247901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:34635 (local address 127.7.24.194:41715)
0620 14:53:01.248313 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:01.248325 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:01.248449 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:01.248737 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:01.248740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:01.248756 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:01.248879 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.248885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.249607 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.249613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.250398 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:01.250407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.251092 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.251096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:01.251191 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.251220 (+ 29us) client_negotiation.cc:769] Sending connection context
0620 14:53:01.251726 (+ 506us) client_negotiation.cc:240] Negotiation successful
0620 14:53:01.251802 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":136,"threads_started":1}
I20260620 14:53:01.251880 8250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "449fb0b7394e4a59bd1e655c5710b58a" candidate_uuid: "722d652632a14df2bbfb2c57aaed2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f0fb65566d4d74bb8068ef48bded0a" is_pre_election: true
I20260620 14:53:01.252238 8250 raft_consensus.cc:2454] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 722d652632a14df2bbfb2c57aaed2786 in term 0.
I20260620 14:53:01.252599 7982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "449fb0b7394e4a59bd1e655c5710b58a" candidate_uuid: "722d652632a14df2bbfb2c57aaed2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54c392e12ff445ba942e0fbd33d9c5ce" is_pre_election: true
I20260620 14:53:01.252767 8049 leader_election.cc:302] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 722d652632a14df2bbfb2c57aaed2786, b7f0fb65566d4d74bb8068ef48bded0a; no voters:
I20260620 14:53:01.252887 7982 raft_consensus.cc:2454] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 722d652632a14df2bbfb2c57aaed2786 in term 0.
I20260620 14:53:01.253054 8320 raft_consensus.cc:2781] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:01.253180 8320 raft_consensus.cc:482] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:01.253242 8320 raft_consensus.cc:3037] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:01.254292 8320 raft_consensus.cc:504] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f0fb65566d4d74bb8068ef48bded0a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 35423 } } peers { permanent_uuid: "54c392e12ff445ba942e0fbd33d9c5ce" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34635 } } peers { permanent_uuid: "722d652632a14df2bbfb2c57aaed2786" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38613 } }
I20260620 14:53:01.254663 8320 leader_election.cc:288] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [CANDIDATE]: Term 1 election: Requested vote from peers b7f0fb65566d4d74bb8068ef48bded0a (127.7.24.195:35423), 54c392e12ff445ba942e0fbd33d9c5ce (127.7.24.193:34635)
I20260620 14:53:01.255071 8250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "449fb0b7394e4a59bd1e655c5710b58a" candidate_uuid: "722d652632a14df2bbfb2c57aaed2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f0fb65566d4d74bb8068ef48bded0a"
I20260620 14:53:01.255117 7982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "449fb0b7394e4a59bd1e655c5710b58a" candidate_uuid: "722d652632a14df2bbfb2c57aaed2786" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54c392e12ff445ba942e0fbd33d9c5ce"
I20260620 14:53:01.255175 8250 raft_consensus.cc:3037] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:01.255267 7982 raft_consensus.cc:3037] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:01.256500 8250 raft_consensus.cc:2454] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 722d652632a14df2bbfb2c57aaed2786 in term 1.
I20260620 14:53:01.256668 7982 raft_consensus.cc:2454] T 449fb0b7394e4a59bd1e655c5710b58a P 54c392e12ff445ba942e0fbd33d9c5ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 722d652632a14df2bbfb2c57aaed2786 in term 1.
I20260620 14:53:01.256985 8049 leader_election.cc:302] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 722d652632a14df2bbfb2c57aaed2786, b7f0fb65566d4d74bb8068ef48bded0a; no voters:
I20260620 14:53:01.257190 8320 raft_consensus.cc:2781] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:01.257450 8320 raft_consensus.cc:686] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [term 1 LEADER]: Becoming Leader. State: Replica: 722d652632a14df2bbfb2c57aaed2786, State: Running, Role: LEADER
I20260620 14:53:01.257791 8320 consensus_queue.cc:237] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f0fb65566d4d74bb8068ef48bded0a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 35423 } } peers { permanent_uuid: "54c392e12ff445ba942e0fbd33d9c5ce" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34635 } } peers { permanent_uuid: "722d652632a14df2bbfb2c57aaed2786" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38613 } }
I20260620 14:53:01.261153 7837 catalog_manager.cc:5495] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786 reported cstate change: term changed from 0 to 1, leader changed from <none> to 722d652632a14df2bbfb2c57aaed2786 (127.7.24.194). New cstate: current_term: 1 leader_uuid: "722d652632a14df2bbfb2c57aaed2786" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f0fb65566d4d74bb8068ef48bded0a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 35423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54c392e12ff445ba942e0fbd33d9c5ce" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "722d652632a14df2bbfb2c57aaed2786" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38613 } health_report { overall_health: HEALTHY } } }
Jun 20 14:53:01 dist-test-slave-sms9 krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:53:01.284921 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.275240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44392 (local address 127.7.24.193:34635)
0620 14:53:01.275474 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:01.275479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:01.275608 (+ 129us) server_negotiation.cc:407] Connection header received
0620 14:53:01.275780 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:01.275784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:01.275839 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:01.275945 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:01.276662 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.277241 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.278145 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.278382 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.278530 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.281112 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:01.281127 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:01.281143 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:01.281168 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:01.283207 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:01.283646 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:01.283652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:01.283653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:01.283737 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:01.284034 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:01.284036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:01.284038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:01.284289 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:01.284445 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:01.284675 (+ 230us) server_negotiation.cc:299] Negotiation successful
0620 14:53:01.284780 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
Jun 20 14:53:01 dist-test-slave-sms9 krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967181, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 14:53:01 dist-test-slave-sms9 krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967181, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:53:01.315124 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.305020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.7.24.193:34635)
0620 14:53:01.305175 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:01.305181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:01.305356 (+ 175us) server_negotiation.cc:407] Connection header received
0620 14:53:01.305504 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:01.305507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:01.305551 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:01.305624 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:01.306506 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.307071 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.308251 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.308492 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.308617 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.311540 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:01.311544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:01.311547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:01.311577 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:01.313256 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:01.313727 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:01.313730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:01.313732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:01.313822 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:01.314138 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:01.314141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:01.314143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:01.314362 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:01.314533 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:01.314849 (+ 316us) server_negotiation.cc:299] Negotiation successful
0620 14:53:01.314963 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260620 14:53:01.321626 8297 tablet.cc:2369] T 449fb0b7394e4a59bd1e655c5710b58a P b7f0fb65566d4d74bb8068ef48bded0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 14:53:01 dist-test-slave-sms9 krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 14:53:01 dist-test-slave-sms9 krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:53:01.343873 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:01.334165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44410 (local address 127.7.24.193:34635)
0620 14:53:01.334379 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:01.334384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:01.334401 (+ 17us) server_negotiation.cc:407] Connection header received
0620 14:53:01.334604 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:01.334607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:01.334655 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:01.334736 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:01.335504 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.336103 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.337113 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:01.337349 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:01.337459 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:01.340443 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:01.340447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:01.340449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:01.340480 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:01.342074 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:01.342660 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:01.342664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:01.342665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:01.342751 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:01.343091 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:01.343093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:01.343102 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:01.343292 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:01.343392 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:01.343636 (+ 244us) server_negotiation.cc:299] Negotiation successful
0620 14:53:01.343744 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260620 14:53:01.345304 7942 tablet_service.cc:2908] Rejecting scan request for tablet 449fb0b7394e4a59bd1e655c5710b58a: Uninitialized: clean time has not yet been initialized
W20260620 14:53:01.346047 8163 tablet.cc:2369] T 449fb0b7394e4a59bd1e655c5710b58a P 722d652632a14df2bbfb2c57aaed2786: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:53:01.347474 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 7899
I20260620 14:53:01.353385 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 8033
I20260620 14:53:01.358706 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 8167
I20260620 14:53:01.364238 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 7805
2026-06-20T14:53:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2446 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:01 dist-test-slave-sms9 krb5kdc[8337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:01 dist-test-slave-sms9 krb5kdc[8337](info): set up 2 sockets
Jun 20 14:53:01 dist-test-slave-sms9 krb5kdc[8337](info): commencing operation
krb5kdc: starting...
W20260620 14:53:02.430359 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:02 dist-test-slave-sms9 krb5kdc[8337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:02.566540 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40735
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:40735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:02.685339 8350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:02.685593 8350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:02.685632 8350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:02.689697 8350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:02.689756 8350 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 14:53:02.689774 8350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:02.689791 8350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:02.689807 8350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:02.689821 8350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:02.694967 8350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:40735
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:02.695935 8350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:02.696872 8350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:02.705678 8358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:02.705839 8350 server_base.cc:1034] running on GCE node
W20260620 14:53:02.705669 8356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:02.705673 8355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:02.706523 8350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:02.707785 8350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:02.709012 8350 hybrid_clock.cc:648] HybridClock initialized: now 1781967182708992 us; error 31 us; skew 500 ppm
Jun 20 14:53:02 dist-test-slave-sms9 krb5kdc[8337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:02.716238 8350 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:02.717160 8350 webserver.cc:460] Webserver started at http://127.7.24.254:34533/ using document root <none> and password file <none>
I20260620 14:53:02.717731 8350 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:02.717824 8350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:02.718063 8350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:02.720196 8350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "6e28baffb5a34549a896555398674ee8"
format_stamp: "Formatted at 2026-06-20 14:53:02 on dist-test-slave-sms9"
server_key: "f028d945ba0d4933116f41ce1788a11a"
server_key_iv: "495ab70e4a3420bb316dbbfbb732a995"
server_key_version: "encryptionkey@0"
I20260620 14:53:02.720768 8350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "6e28baffb5a34549a896555398674ee8"
format_stamp: "Formatted at 2026-06-20 14:53:02 on dist-test-slave-sms9"
server_key: "f028d945ba0d4933116f41ce1788a11a"
server_key_iv: "495ab70e4a3420bb316dbbfbb732a995"
server_key_version: "encryptionkey@0"
I20260620 14:53:02.724519 8350 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260620 14:53:02.727095 8365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:02.728327 8350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:02.728479 8350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "6e28baffb5a34549a896555398674ee8"
format_stamp: "Formatted at 2026-06-20 14:53:02 on dist-test-slave-sms9"
server_key: "f028d945ba0d4933116f41ce1788a11a"
server_key_iv: "495ab70e4a3420bb316dbbfbb732a995"
server_key_version: "encryptionkey@0"
I20260620 14:53:02.728592 8350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:02.756371 8350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:02.759548 8350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:02.759766 8350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:02.768036 8350 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:40735
I20260620 14:53:02.768062 8427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:40735 every 8 connection(s)
I20260620 14:53:02.769043 8350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:02.771734 8428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:02.774230 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 8350
I20260620 14:53:02.774331 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:02.774638 7267 external_mini_cluster.cc:1428] Setting key da02f36f902763193b456be43da28b30
I20260620 14:53:02.777904 8428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8: Bootstrap starting.
I20260620 14:53:02.780264 8428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:02.781016 8428 log.cc:824] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:02.783061 8428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8: No bootstrap required, opened a new log
Jun 20 14:53:02 dist-test-slave-sms9 krb5kdc[8337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:02.787825 8428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e28baffb5a34549a896555398674ee8" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40735 } }
I20260620 14:53:02.788154 8428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:02.788221 8428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e28baffb5a34549a896555398674ee8, State: Initialized, Role: FOLLOWER
I20260620 14:53:02.788700 8428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e28baffb5a34549a896555398674ee8" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40735 } }
I20260620 14:53:02.788873 8428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:02.788945 8428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:02.789045 8428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:02.789597 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:02.776153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58710 (local address 127.7.24.254:40735)
0620 14:53:02.776611 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:02.776626 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:02.776661 (+ 35us) server_negotiation.cc:407] Connection header received
0620 14:53:02.777291 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:02.777308 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:02.777646 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:02.778005 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:02.778812 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:02.779803 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:02.780740 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:02.781268 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:02.781451 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:02.784081 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:02.784088 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:02.784104 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:02.784134 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:02.786175 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:02.786843 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:02.786850 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:02.786856 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:02.787022 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:02.787350 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:02.787353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:02.787354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:02.787865 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:02.788061 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:02.788522 (+ 461us) server_negotiation.cc:299] Negotiation successful
0620 14:53:02.788831 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":156,"threads_started":1}
I20260620 14:53:02.790133 8428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e28baffb5a34549a896555398674ee8" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40735 } }
I20260620 14:53:02.790535 8428 leader_election.cc:302] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e28baffb5a34549a896555398674ee8; no voters:
I20260620 14:53:02.790915 8428 leader_election.cc:288] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:02.791029 8433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:02.791270 8433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [term 1 LEADER]: Becoming Leader. State: Replica: 6e28baffb5a34549a896555398674ee8, State: Running, Role: LEADER
I20260620 14:53:02.791577 8433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e28baffb5a34549a896555398674ee8" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40735 } }
I20260620 14:53:02.792181 8428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:02.793484 8435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e28baffb5a34549a896555398674ee8. Latest consensus state: current_term: 1 leader_uuid: "6e28baffb5a34549a896555398674ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e28baffb5a34549a896555398674ee8" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40735 } } }
I20260620 14:53:02.793537 8434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e28baffb5a34549a896555398674ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e28baffb5a34549a896555398674ee8" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40735 } } }
I20260620 14:53:02.793609 8435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:02.793642 8434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:02.793972 8443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:02.797271 8443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:02.803177 8443 catalog_manager.cc:1347] Generated new cluster ID: efb33300cc254018bfe3e27d0b9d4117
I20260620 14:53:02.803258 8443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:02.818723 8443 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:02.819967 8443 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:02.835218 8443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8: Generated new TSK 0
I20260620 14:53:02.836112 8443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:53:02.867089 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40735
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:02.978473 8454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:02.978729 8454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:02.978773 8454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:02.983356 8454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:02.983409 8454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:02.983480 8454 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 14:53:02.983542 8454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:02.988874 8454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.24.254:40735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:02.989943 8454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:02.990975 8454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:03.000968 8459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:03.000968 8462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:03.000985 8460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:03.001446 8454 server_base.cc:1034] running on GCE node
I20260620 14:53:03.001852 8454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:03.002636 8454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:03.003859 8454 hybrid_clock.cc:648] HybridClock initialized: now 1781967183003842 us; error 31 us; skew 500 ppm
Jun 20 14:53:02 dist-test-slave-sms9 krb5kdc[8337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:03.006948 8454 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:53:03.007741 8454 webserver.cc:460] Webserver started at http://127.7.24.193:37757/ using document root <none> and password file <none>
I20260620 14:53:03.008256 8454 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:03.008316 8454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:03.008478 8454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:03.010452 8454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "48533dcce43c427ab6c6f23854fc9025"
format_stamp: "Formatted at 2026-06-20 14:53:03 on dist-test-slave-sms9"
server_key: "79529d510bcafd8427437f05665bc722"
server_key_iv: "2523031c184c09968658208acd7d6c41"
server_key_version: "encryptionkey@0"
I20260620 14:53:03.011107 8454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "48533dcce43c427ab6c6f23854fc9025"
format_stamp: "Formatted at 2026-06-20 14:53:03 on dist-test-slave-sms9"
server_key: "79529d510bcafd8427437f05665bc722"
server_key_iv: "2523031c184c09968658208acd7d6c41"
server_key_version: "encryptionkey@0"
I20260620 14:53:03.015023 8454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 14:53:03.017446 8469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:03.018874 8454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:03.019007 8454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "48533dcce43c427ab6c6f23854fc9025"
format_stamp: "Formatted at 2026-06-20 14:53:03 on dist-test-slave-sms9"
server_key: "79529d510bcafd8427437f05665bc722"
server_key_iv: "2523031c184c09968658208acd7d6c41"
server_key_version: "encryptionkey@0"
I20260620 14:53:03.019127 8454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:03.043680 8454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:03.047204 8454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:03.047425 8454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:03.048899 8454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:03.048946 8454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:03.049011 8454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:03.049046 8454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 14:53:03 dist-test-slave-sms9 krb5kdc[8337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:03.060041 8454 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:36259
I20260620 14:53:03.060119 8584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:36259 every 8 connection(s)
I20260620 14:53:03.060979 8454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:03.063288 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.049867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:40511 (local address 127.7.24.254:40735)
0620 14:53:03.050021 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.050026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.050904 (+ 878us) server_negotiation.cc:407] Connection header received
0620 14:53:03.051996 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.052000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.052057 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.052150 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:03.053806 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.054740 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.056070 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.056319 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.056443 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.059318 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:03.059321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:03.059323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:03.059371 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:03.061049 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.061754 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.061760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.061763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.061867 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.062249 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.062253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.062254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.062500 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:03.062605 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.063060 (+ 455us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.063166 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 14:53:03.063755 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 8454
I20260620 14:53:03.063866 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:03.064145 7267 external_mini_cluster.cc:1428] Setting key 5378b77b21e0d7ae0d69552f4c71ed08
I20260620 14:53:03.064237 8478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.050255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40735 (local address 127.7.24.193:40511)
0620 14:53:03.050728 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.050777 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.051745 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.052376 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.052387 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.052877 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:03.053619 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.053642 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.054913 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.054915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.055945 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.055955 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.056494 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.056498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.056671 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.057282 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:03.057298 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:03.059120 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:03.061208 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.061213 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.061228 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.061597 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.061969 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.061972 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.061974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.062132 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.062613 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:03.062618 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:03.062841 (+ 223us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.062995 (+ 154us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.063248 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":158,"threads_started":1}
I20260620 14:53:03.065673 8585 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40735
I20260620 14:53:03.065832 8585 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:03.066421 8585 heartbeater.cc:511] Master 127.7.24.254:40735 requested a full tablet report, sending...
I20260620 14:53:03.068113 8382 ts_manager.cc:194] Registered new tserver with Master: 48533dcce43c427ab6c6f23854fc9025 (127.7.24.193:36259)
I20260620 14:53:03.069533 8382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:40511
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:53:03.075665 8592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.067885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40735 (local address 127.7.24.193:43893)
0620 14:53:03.068213 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.068230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.068334 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.068585 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.068591 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.068783 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:03.068886 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.068891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.069882 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.069889 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.070680 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.070689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.071143 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.071147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.071276 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.071864 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:03.071876 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:03.072211 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:03.074120 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.074124 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.074126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.074340 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.074660 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.074664 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.074665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.074760 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.075140 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:03.075143 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:03.075263 (+ 120us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.075385 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.075510 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":28,"thread_start_us":114,"threads_started":1}
I20260620 14:53:03.075788 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.068213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:43893 (local address 127.7.24.254:40735)
0620 14:53:03.068317 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.068321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.068335 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:03.068427 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.068429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.068467 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.068547 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:03.069003 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.069720 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.070803 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.071029 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.071142 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.072358 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:03.072362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:03.072365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:03.072388 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:03.073982 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.074448 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.074452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.074454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.074561 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.074869 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.074872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.074874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.075043 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:03.075151 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.075482 (+ 331us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.075615 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:53:03.093295 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40735
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:03.203397 8594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:03.203658 8594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:03.203703 8594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:03.207723 8594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:03.207791 8594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:03.207832 8594 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 14:53:03.207865 8594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:03.213231 8594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.24.254:40735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:03.214228 8594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:03.215273 8594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:03.225222 8599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:03.225350 8594 server_base.cc:1034] running on GCE node
W20260620 14:53:03.225170 8600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:03.225515 8602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:03.225951 8594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:03.226723 8594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:03.227967 8594 hybrid_clock.cc:648] HybridClock initialized: now 1781967183227938 us; error 46 us; skew 500 ppm
Jun 20 14:53:03 dist-test-slave-sms9 krb5kdc[8337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:03.230752 8594 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:53:03.231725 8594 webserver.cc:460] Webserver started at http://127.7.24.194:38069/ using document root <none> and password file <none>
I20260620 14:53:03.232327 8594 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:03.232417 8594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:03.232681 8594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:03.234858 8594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "0d4266dc82964a6c828b6ac739b93510"
format_stamp: "Formatted at 2026-06-20 14:53:03 on dist-test-slave-sms9"
server_key: "51d047594d6ee3f2f85bee4bb5375a94"
server_key_iv: "d2df881f03b3b8c5783178810f498093"
server_key_version: "encryptionkey@0"
I20260620 14:53:03.235424 8594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "0d4266dc82964a6c828b6ac739b93510"
format_stamp: "Formatted at 2026-06-20 14:53:03 on dist-test-slave-sms9"
server_key: "51d047594d6ee3f2f85bee4bb5375a94"
server_key_iv: "d2df881f03b3b8c5783178810f498093"
server_key_version: "encryptionkey@0"
I20260620 14:53:03.239365 8594 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 14:53:03.241734 8609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:03.243021 8594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:03.243155 8594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "0d4266dc82964a6c828b6ac739b93510"
format_stamp: "Formatted at 2026-06-20 14:53:03 on dist-test-slave-sms9"
server_key: "51d047594d6ee3f2f85bee4bb5375a94"
server_key_iv: "d2df881f03b3b8c5783178810f498093"
server_key_version: "encryptionkey@0"
I20260620 14:53:03.243261 8594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:03.260375 8594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:03.263944 8594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:03.264156 8594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:03.265772 8594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:03.265956 8594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:03.266067 8594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:03.266105 8594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 14:53:03 dist-test-slave-sms9 krb5kdc[8337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:03.277335 8594 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:34429
I20260620 14:53:03.277444 8724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:34429 every 8 connection(s)
I20260620 14:53:03.278329 8594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:03.279803 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 8594
I20260620 14:53:03.279910 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:03.280182 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.266745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:58579 (local address 127.7.24.254:40735)
0620 14:53:03.266892 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.266898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.267897 (+ 999us) server_negotiation.cc:407] Connection header received
0620 14:53:03.268907 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.268913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.268978 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.269045 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:03.270622 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.271197 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.272341 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.272613 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.272741 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.276019 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:03.276023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:03.276026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:03.276053 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:03.277864 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.278495 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.278499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.278501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.278600 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.279022 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.279027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.279030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.279272 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:03.279384 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.279874 (+ 490us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.280011 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 14:53:03.280202 7267 external_mini_cluster.cc:1428] Setting key 7bfa6d736744c9d8d271c4619f1d70be
I20260620 14:53:03.280992 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.267266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40735 (local address 127.7.24.194:58579)
0620 14:53:03.267722 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.267772 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.268634 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.269407 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.269415 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.269835 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:03.270429 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.270453 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.271333 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.271338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.272211 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.272220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.272783 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.272787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.272966 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.273771 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:03.273796 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:03.275823 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:03.277975 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.277983 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.277996 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.278324 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.278713 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.278717 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.278719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.278896 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.279379 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:03.279384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:03.279609 (+ 225us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.279776 (+ 167us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.280114 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":138,"threads_started":1}
I20260620 14:53:03.282243 8725 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40735
I20260620 14:53:03.282413 8725 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:03.282922 8725 heartbeater.cc:511] Master 127.7.24.254:40735 requested a full tablet report, sending...
I20260620 14:53:03.283995 8380 ts_manager.cc:194] Registered new tserver with Master: 0d4266dc82964a6c828b6ac739b93510 (127.7.24.194:34429)
I20260620 14:53:03.284629 8380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:58579
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:53:03.293464 8732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.284630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40735 (local address 127.7.24.194:49687)
0620 14:53:03.285042 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.285059 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.285196 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.285569 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.285571 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.285824 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:03.285977 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.285988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.287156 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.287162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.288209 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.288225 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.288677 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.288683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.288833 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.289393 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:03.289411 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:03.289866 (+ 455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:03.291697 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.291703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.291708 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.291907 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.292245 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.292248 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.292250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.292330 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.292801 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:03.292804 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:03.292933 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.293067 (+ 134us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.293231 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":105,"threads_started":1}
I20260620 14:53:03.293473 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.284677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:49687 (local address 127.7.24.254:40735)
0620 14:53:03.284783 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.284787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.285126 (+ 339us) server_negotiation.cc:407] Connection header received
0620 14:53:03.285301 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.285307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.285356 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.285407 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:03.286112 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.286992 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.288336 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.288547 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.288633 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.290037 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:03.290040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:03.290042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:03.290063 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:03.291590 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.292042 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.292047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.292049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.292136 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.292430 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.292435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.292444 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.292663 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:03.292783 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.293173 (+ 390us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.293325 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:53:03.310173 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40735
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:03.423225 8734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:03.423499 8734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:03.423545 8734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:03.427863 8734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:03.427938 8734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:03.427981 8734 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 14:53:03.428016 8734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:03.433528 8734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.24.254:40735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:03.434582 8734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:03.435549 8734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:03.445302 8739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:03.445335 8740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:03.445418 8734 server_base.cc:1034] running on GCE node
W20260620 14:53:03.445303 8742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:03.445999 8734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:03.446724 8734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:03.447899 8734 hybrid_clock.cc:648] HybridClock initialized: now 1781967183447868 us; error 43 us; skew 500 ppm
Jun 20 14:53:03 dist-test-slave-sms9 krb5kdc[8337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:03.450567 8734 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:53:03.451443 8734 webserver.cc:460] Webserver started at http://127.7.24.195:42639/ using document root <none> and password file <none>
I20260620 14:53:03.451987 8734 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:03.452072 8734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:03.452293 8734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:03.454361 8734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "14747d1bec724c76804fd1d56b85bb37"
format_stamp: "Formatted at 2026-06-20 14:53:03 on dist-test-slave-sms9"
server_key: "86a3fdbfdd962d9cb24ab187dff11431"
server_key_iv: "c022a779e293dbc66cc9eec7eeab300b"
server_key_version: "encryptionkey@0"
I20260620 14:53:03.454973 8734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "14747d1bec724c76804fd1d56b85bb37"
format_stamp: "Formatted at 2026-06-20 14:53:03 on dist-test-slave-sms9"
server_key: "86a3fdbfdd962d9cb24ab187dff11431"
server_key_iv: "c022a779e293dbc66cc9eec7eeab300b"
server_key_version: "encryptionkey@0"
I20260620 14:53:03.458729 8734 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:03.461120 8749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:03.462311 8734 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:03.462527 8734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "14747d1bec724c76804fd1d56b85bb37"
format_stamp: "Formatted at 2026-06-20 14:53:03 on dist-test-slave-sms9"
server_key: "86a3fdbfdd962d9cb24ab187dff11431"
server_key_iv: "c022a779e293dbc66cc9eec7eeab300b"
server_key_version: "encryptionkey@0"
I20260620 14:53:03.462642 8734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:03.473389 8734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:03.476423 8734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:03.476626 8734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:03.478075 8734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:03.478145 8734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:03.478215 8734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:03.478255 8734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:03.488056 8734 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:41139
Jun 20 14:53:03 dist-test-slave-sms9 krb5kdc[8337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:03.488775 8864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:41139 every 8 connection(s)
I20260620 14:53:03.488994 8734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:03.492419 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.478855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:46889 (local address 127.7.24.254:40735)
0620 14:53:03.479061 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.479066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.480003 (+ 937us) server_negotiation.cc:407] Connection header received
0620 14:53:03.481404 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.481410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.481470 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.481584 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:03.483096 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.484013 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.485657 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.486036 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.486160 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.489009 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:03.489013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:03.489016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:03.489049 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:03.490639 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.491093 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.491095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.491096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.491170 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.491468 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.491470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.491471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.491632 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:03.491768 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.492198 (+ 430us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.492293 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260620 14:53:03.493259 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.479342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40735 (local address 127.7.24.195:46889)
0620 14:53:03.479814 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.479871 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.481014 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.481757 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.481766 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.482199 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:03.482919 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.482939 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.484153 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.484157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.485526 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.485536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.486205 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.486210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.486447 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.487161 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:03.487184 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:03.488840 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:03.490737 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.490741 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.490751 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.491010 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.491247 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.491249 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.491250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.491368 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.491779 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:03.491785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:03.492004 (+ 219us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.492152 (+ 148us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.492370 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":110,"threads_started":1}
I20260620 14:53:03.494690 8865 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40735
I20260620 14:53:03.494839 8865 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:03.495314 8865 heartbeater.cc:511] Master 127.7.24.254:40735 requested a full tablet report, sending...
I20260620 14:53:03.496639 8380 ts_manager.cc:194] Registered new tserver with Master: 14747d1bec724c76804fd1d56b85bb37 (127.7.24.195:41139)
I20260620 14:53:03.496942 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 8734
I20260620 14:53:03.497105 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:03.497400 8380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:46889
I20260620 14:53:03.497510 7267 external_mini_cluster.cc:1428] Setting key ac89d795f7bc07b698609badf5db3e1b
I20260620 14:53:03.499862 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:53:03.509775 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.502153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:55715 (local address 127.7.24.254:40735)
0620 14:53:03.502285 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.502292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.502575 (+ 283us) server_negotiation.cc:407] Connection header received
0620 14:53:03.502768 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.502770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.502810 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.502877 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:03.503462 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.504002 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.504925 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.505188 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.505299 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.506438 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:03.506441 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:03.506444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:03.506500 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:03.508010 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.508529 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.508532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.508534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.508618 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.508944 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.508947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.508949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.509126 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:03.509212 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.509562 (+ 350us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.509659 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:53:03.509953 8872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.501809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40735 (local address 127.7.24.195:55715)
0620 14:53:03.502492 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.502510 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.502650 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.502925 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.502930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.503184 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:03.503323 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.503340 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.504127 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.504130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.504804 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.504813 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.505319 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.505323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.505408 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.505951 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:03.505971 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:03.506286 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:03.508124 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.508130 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.508135 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.508415 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.508713 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.508718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.508721 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.508819 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.509236 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:03.509240 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:03.509358 (+ 118us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.509489 (+ 131us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.509656 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":159,"threads_started":1}
Jun 20 14:53:03 dist-test-slave-sms9 krb5kdc[8337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967183, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 14:53:03 dist-test-slave-sms9 krb5kdc[8337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967183, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:03.526305 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.518533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58714 (local address 127.7.24.254:40735)
0620 14:53:03.518660 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.518663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.518673 (+ 10us) server_negotiation.cc:407] Connection header received
0620 14:53:03.518705 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.518707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.518740 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.518837 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:03.519503 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.520014 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.520851 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.521056 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.521166 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.523631 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:03.523634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:03.523636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:03.523657 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:03.524807 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.525220 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.525222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.525224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.525291 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.525509 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.525511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.525512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.525666 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:03.525762 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.526095 (+ 333us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.526185 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:53:03.529419 8380 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:03.532488 8380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:03.547017 8883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.541313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:36259 (local address 127.0.0.1:40556)
0620 14:53:03.541897 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.541914 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.542000 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.542520 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.542523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.542539 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:03.542797 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.542807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.543860 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.543867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.545309 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.545319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.546406 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.546410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.546549 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.546606 (+ 57us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.546708 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.546857 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":325,"thread_start_us":102,"threads_started":1}
I20260620 14:53:03.547199 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.540774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54640 (local address 127.7.24.195:41139)
0620 14:53:03.541183 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.541197 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.541990 (+ 793us) server_negotiation.cc:407] Connection header received
0620 14:53:03.542077 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.542085 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.542266 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.542434 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:03.542878 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.544039 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.545792 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.546517 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.546614 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.546762 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.546947 (+ 185us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.547045 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":116,"threads_started":1}
I20260620 14:53:03.547530 8887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.541529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.7.24.193:36259)
0620 14:53:03.541942 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.541959 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.541981 (+ 22us) server_negotiation.cc:407] Connection header received
0620 14:53:03.542123 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.542127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.542269 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.542418 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:03.542939 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.543722 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.545443 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.546272 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.546393 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.546594 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.547245 (+ 651us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.547364 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":89,"threads_started":1}
I20260620 14:53:03.547803 8882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.540489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:34429 (local address 127.0.0.1:58840)
0620 14:53:03.541602 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.541652 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.541871 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.542584 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.542586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.542599 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:03.542762 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.542767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.544274 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.544277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.545629 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.545640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.547043 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.547049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.547152 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.547175 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.547548 (+ 373us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.547638 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":69,"thread_start_us":90,"threads_started":1}
I20260620 14:53:03.547782 8886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.540611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:41139 (local address 127.0.0.1:54640)
0620 14:53:03.541682 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.541698 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.541831 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.542429 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.542436 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.542484 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:03.542731 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.542741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.544164 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.544168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.545629 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.545640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.546680 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.546684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.546814 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.546838 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.546918 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.547003 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1000,"mutex_wait_us":112,"thread_start_us":127,"threads_started":1}
I20260620 14:53:03.548388 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.540612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58840 (local address 127.7.24.194:34429)
0620 14:53:03.541266 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.541279 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.541826 (+ 547us) server_negotiation.cc:407] Connection header received
0620 14:53:03.542032 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.542061 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.542264 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.542418 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:03.542910 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.544164 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.545761 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.546700 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.546810 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.546975 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.548127 (+ 1152us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.548232 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":117,"threads_started":1}
I20260620 14:53:03.550592 8519 tablet_service.cc:1467] Processing CreateTablet for tablet 3166626c5f3c4178a386a30cfce37892 (DEFAULT_TABLE table=test-table [id=51624938cb7f45d79a0d7a8b2c04f06f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:03.550725 8799 tablet_service.cc:1467] Processing CreateTablet for tablet 3166626c5f3c4178a386a30cfce37892 (DEFAULT_TABLE table=test-table [id=51624938cb7f45d79a0d7a8b2c04f06f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:03.551076 8657 tablet_service.cc:1467] Processing CreateTablet for tablet 3166626c5f3c4178a386a30cfce37892 (DEFAULT_TABLE table=test-table [id=51624938cb7f45d79a0d7a8b2c04f06f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:03.551644 8519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3166626c5f3c4178a386a30cfce37892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:03.551818 8799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3166626c5f3c4178a386a30cfce37892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:03.551993 8657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3166626c5f3c4178a386a30cfce37892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:03.558254 8888 tablet_bootstrap.cc:492] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025: Bootstrap starting.
I20260620 14:53:03.559711 8889 tablet_bootstrap.cc:492] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37: Bootstrap starting.
I20260620 14:53:03.560329 8890 tablet_bootstrap.cc:492] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510: Bootstrap starting.
I20260620 14:53:03.561442 8888 tablet_bootstrap.cc:654] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:03.562235 8890 tablet_bootstrap.cc:654] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:03.562304 8888 log.cc:824] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:03.563031 8890 log.cc:824] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:03.564429 8888 tablet_bootstrap.cc:492] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025: No bootstrap required, opened a new log
I20260620 14:53:03.564478 8889 tablet_bootstrap.cc:654] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:03.564651 8888 ts_tablet_manager.cc:1397] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260620 14:53:03.564990 8890 tablet_bootstrap.cc:492] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510: No bootstrap required, opened a new log
I20260620 14:53:03.565171 8890 ts_tablet_manager.cc:1397] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 14:53:03.565251 8889 log.cc:824] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:03.567148 8889 tablet_bootstrap.cc:492] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37: No bootstrap required, opened a new log
I20260620 14:53:03.567340 8889 ts_tablet_manager.cc:1397] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260620 14:53:03.568928 8888 raft_consensus.cc:348] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } }
I20260620 14:53:03.569177 8888 raft_consensus.cc:374] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:03.569248 8888 raft_consensus.cc:729] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48533dcce43c427ab6c6f23854fc9025, State: Initialized, Role: FOLLOWER
I20260620 14:53:03.569280 8890 raft_consensus.cc:348] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } }
I20260620 14:53:03.569526 8890 raft_consensus.cc:374] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:03.569581 8890 raft_consensus.cc:729] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d4266dc82964a6c828b6ac739b93510, State: Initialized, Role: FOLLOWER
I20260620 14:53:03.569746 8888 consensus_queue.cc:260] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } }
I20260620 14:53:03.569972 8890 consensus_queue.cc:260] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } }
I20260620 14:53:03.570493 8585 heartbeater.cc:503] Master 127.7.24.254:40735 was elected leader, sending a full tablet report...
I20260620 14:53:03.570891 8888 ts_tablet_manager.cc:1428] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260620 14:53:03.570912 8725 heartbeater.cc:503] Master 127.7.24.254:40735 was elected leader, sending a full tablet report...
I20260620 14:53:03.572024 8890 ts_tablet_manager.cc:1428] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260620 14:53:03.572150 8895 raft_consensus.cc:482] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:03.572022 8889 raft_consensus.cc:348] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } }
I20260620 14:53:03.572255 8889 raft_consensus.cc:374] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:03.572356 8889 raft_consensus.cc:729] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14747d1bec724c76804fd1d56b85bb37, State: Initialized, Role: FOLLOWER
I20260620 14:53:03.572799 8889 consensus_queue.cc:260] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } }
I20260620 14:53:03.572831 8895 raft_consensus.cc:504] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } }
I20260620 14:53:03.573606 8889 ts_tablet_manager.cc:1428] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 14:53:03.573836 8865 heartbeater.cc:503] Master 127.7.24.254:40735 was elected leader, sending a full tablet report...
I20260620 14:53:03.574033 8895 leader_election.cc:288] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14747d1bec724c76804fd1d56b85bb37 (127.7.24.195:41139), 48533dcce43c427ab6c6f23854fc9025 (127.7.24.193:36259)
I20260620 14:53:03.578568 8887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.574353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:59781 (local address 127.7.24.193:36259)
0620 14:53:03.574623 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.574629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.574647 (+ 18us) server_negotiation.cc:407] Connection header received
0620 14:53:03.574693 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.574698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.574770 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.574846 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:03.575244 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.575960 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.577425 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.577979 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.578119 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.578173 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.578365 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.578432 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260620 14:53:03.578568 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.574251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:36259 (local address 127.7.24.194:59781)
0620 14:53:03.574425 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.574441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.574543 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.574968 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.574971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.574990 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:03.575103 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.575111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.576083 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.576086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.577263 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.577273 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.578108 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.578112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.578208 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.578230 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.578339 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.578414 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260620 14:53:03.579149 8539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3166626c5f3c4178a386a30cfce37892" candidate_uuid: "0d4266dc82964a6c828b6ac739b93510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48533dcce43c427ab6c6f23854fc9025" is_pre_election: true
I20260620 14:53:03.579437 8539 raft_consensus.cc:2454] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d4266dc82964a6c828b6ac739b93510 in term 0.
I20260620 14:53:03.579911 8610 leader_election.cc:302] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d4266dc82964a6c828b6ac739b93510, 48533dcce43c427ab6c6f23854fc9025; no voters:
I20260620 14:53:03.580190 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.574251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:41139 (local address 127.7.24.194:59435)
0620 14:53:03.575745 (+ 1494us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.575760 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.575893 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.576280 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.576283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.576299 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:03.576917 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.576929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.577841 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.577844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.578735 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.578745 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.579743 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.579747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.579818 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.579883 (+ 65us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.579949 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.580002 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1416,"mutex_wait_us":543,"thread_start_us":109,"threads_started":1}
I20260620 14:53:03.580188 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.574489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:59435 (local address 127.7.24.195:41139)
0620 14:53:03.574718 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.574723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.575833 (+ 1110us) server_negotiation.cc:407] Connection header received
0620 14:53:03.576010 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.576014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.576061 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.576110 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:03.577048 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.577713 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.578916 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.579481 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.579605 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.579651 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.579997 (+ 346us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.580075 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260620 14:53:03.580235 8895 raft_consensus.cc:2781] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:03.580386 8895 raft_consensus.cc:482] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:03.580471 8895 raft_consensus.cc:3037] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:03.580770 8819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3166626c5f3c4178a386a30cfce37892" candidate_uuid: "0d4266dc82964a6c828b6ac739b93510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14747d1bec724c76804fd1d56b85bb37" is_pre_election: true
I20260620 14:53:03.581030 8819 raft_consensus.cc:2454] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d4266dc82964a6c828b6ac739b93510 in term 0.
I20260620 14:53:03.581708 8895 raft_consensus.cc:504] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } }
I20260620 14:53:03.582087 8895 leader_election.cc:288] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [CANDIDATE]: Term 1 election: Requested vote from peers 14747d1bec724c76804fd1d56b85bb37 (127.7.24.195:41139), 48533dcce43c427ab6c6f23854fc9025 (127.7.24.193:36259)
I20260620 14:53:03.582413 8819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3166626c5f3c4178a386a30cfce37892" candidate_uuid: "0d4266dc82964a6c828b6ac739b93510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14747d1bec724c76804fd1d56b85bb37"
I20260620 14:53:03.582413 8539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3166626c5f3c4178a386a30cfce37892" candidate_uuid: "0d4266dc82964a6c828b6ac739b93510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48533dcce43c427ab6c6f23854fc9025"
I20260620 14:53:03.582557 8819 raft_consensus.cc:3037] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:03.582557 8539 raft_consensus.cc:3037] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:03.583837 8539 raft_consensus.cc:2454] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d4266dc82964a6c828b6ac739b93510 in term 1.
I20260620 14:53:03.583837 8819 raft_consensus.cc:2454] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d4266dc82964a6c828b6ac739b93510 in term 1.
I20260620 14:53:03.584173 8611 leader_election.cc:302] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d4266dc82964a6c828b6ac739b93510, 14747d1bec724c76804fd1d56b85bb37; no voters:
I20260620 14:53:03.584367 8895 raft_consensus.cc:2781] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:03.584606 8895 raft_consensus.cc:686] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 1 LEADER]: Becoming Leader. State: Replica: 0d4266dc82964a6c828b6ac739b93510, State: Running, Role: LEADER
I20260620 14:53:03.584954 8895 consensus_queue.cc:237] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } }
I20260620 14:53:03.588234 8382 catalog_manager.cc:5495] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d4266dc82964a6c828b6ac739b93510 (127.7.24.194). New cstate: current_term: 1 leader_uuid: "0d4266dc82964a6c828b6ac739b93510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } health_report { overall_health: UNKNOWN } } }
I20260620 14:53:03.607407 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.603788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58854 (local address 127.7.24.194:34429)
0620 14:53:03.603913 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.603917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.604025 (+ 108us) server_negotiation.cc:407] Connection header received
0620 14:53:03.604198 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.604200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.604254 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.604340 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:03.604618 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.605136 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.606049 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.606263 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.606350 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.606707 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:03.607078 (+ 371us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:03.607165 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.607251 (+ 86us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.607302 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 14:53:03.613000 8819 raft_consensus.cc:1261] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37 [term 1 FOLLOWER]: Refusing update from remote peer 0d4266dc82964a6c828b6ac739b93510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:03.613000 8539 raft_consensus.cc:1261] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025 [term 1 FOLLOWER]: Refusing update from remote peer 0d4266dc82964a6c828b6ac739b93510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:03.613736 8898 consensus_queue.cc:1035] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:03.613929 8895 consensus_queue.cc:1035] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:03.621642 8903 mvcc.cc:205] Tried to move back new op lower bound from 7298937584071974912 to 7298937583966638080. Current Snapshot: MvccSnapshot[applied={T|T < 7298937584071974912}]
I20260620 14:53:03.622524 8901 mvcc.cc:205] Tried to move back new op lower bound from 7298937584071974912 to 7298937583966638080. Current Snapshot: MvccSnapshot[applied={T|T < 7298937584071974912}]
I20260620 14:53:03.623644 8902 mvcc.cc:205] Tried to move back new op lower bound from 7298937584071974912 to 7298937583966638080. Current Snapshot: MvccSnapshot[applied={T|T < 7298937584071974912}]
I20260620 14:53:03.633270 8381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:03.633460 8381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:03.636124 8381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8: Sending DeleteTablet for 3 replicas of tablet 3166626c5f3c4178a386a30cfce37892
I20260620 14:53:03.636931 8657 tablet_service.cc:1514] Processing DeleteTablet for tablet 3166626c5f3c4178a386a30cfce37892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:03 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:58840
I20260620 14:53:03.636965 8799 tablet_service.cc:1514] Processing DeleteTablet for tablet 3166626c5f3c4178a386a30cfce37892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:03 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54640
I20260620 14:53:03.637192 8519 tablet_service.cc:1514] Processing DeleteTablet for tablet 3166626c5f3c4178a386a30cfce37892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:03 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:40556
I20260620 14:53:03.637455 8911 tablet_replica.cc:330] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510: stopping tablet replica
I20260620 14:53:03.637616 8913 tablet_replica.cc:330] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025: stopping tablet replica
I20260620 14:53:03.637751 8911 raft_consensus.cc:2229] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:03.637924 8913 raft_consensus.cc:2229] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:03.638139 8911 raft_consensus.cc:2258] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:03.638218 8913 raft_consensus.cc:2258] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:03.638517 8912 tablet_replica.cc:330] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37: stopping tablet replica
I20260620 14:53:03.638779 8912 raft_consensus.cc:2229] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:03.638859 8381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 14:53:03.639034 8912 raft_consensus.cc:2258] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 14:53:03.639250 8381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:03.639676 8913 ts_tablet_manager.cc:1905] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:03.640245 8912 ts_tablet_manager.cc:1905] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:03.642397 8913 ts_tablet_manager.cc:1918] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:03.642510 8913 log.cc:1196] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/wal/wals/3166626c5f3c4178a386a30cfce37892
I20260620 14:53:03.642861 8913 ts_tablet_manager.cc:1939] T 3166626c5f3c4178a386a30cfce37892 P 48533dcce43c427ab6c6f23854fc9025: Deleting consensus metadata
I20260620 14:53:03.643203 8912 ts_tablet_manager.cc:1918] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:03.643290 8912 log.cc:1196] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/3166626c5f3c4178a386a30cfce37892
I20260620 14:53:03.643585 8912 ts_tablet_manager.cc:1939] T 3166626c5f3c4178a386a30cfce37892 P 14747d1bec724c76804fd1d56b85bb37: Deleting consensus metadata
I20260620 14:53:03.644354 8366 catalog_manager.cc:4841] TS 48533dcce43c427ab6c6f23854fc9025 (127.7.24.193:36259): tablet 3166626c5f3c4178a386a30cfce37892 (table test-table [id=51624938cb7f45d79a0d7a8b2c04f06f]) successfully deleted
I20260620 14:53:03.644620 8367 catalog_manager.cc:4841] TS 14747d1bec724c76804fd1d56b85bb37 (127.7.24.195:41139): tablet 3166626c5f3c4178a386a30cfce37892 (table test-table [id=51624938cb7f45d79a0d7a8b2c04f06f]) successfully deleted
I20260620 14:53:03.645594 8519 tablet_service.cc:1467] Processing CreateTablet for tablet 54d26879ba8f4cfb9f520aa6d9bdcb99 (DEFAULT_TABLE table=test-table [id=1782d9c7ac964402954faa86c6b6178f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:03.645601 8799 tablet_service.cc:1467] Processing CreateTablet for tablet 54d26879ba8f4cfb9f520aa6d9bdcb99 (DEFAULT_TABLE table=test-table [id=1782d9c7ac964402954faa86c6b6178f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:03.645843 8799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54d26879ba8f4cfb9f520aa6d9bdcb99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:03.645860 8519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54d26879ba8f4cfb9f520aa6d9bdcb99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:03.646189 8911 ts_tablet_manager.cc:1905] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:03.647073 8657 tablet_service.cc:1467] Processing CreateTablet for tablet 54d26879ba8f4cfb9f520aa6d9bdcb99 (DEFAULT_TABLE table=test-table [id=1782d9c7ac964402954faa86c6b6178f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:03.647328 8657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54d26879ba8f4cfb9f520aa6d9bdcb99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:03.648118 8889 tablet_bootstrap.cc:492] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37: Bootstrap starting.
I20260620 14:53:03.649048 8889 tablet_bootstrap.cc:654] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:03.649226 8911 ts_tablet_manager.cc:1918] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:03.649384 8911 log.cc:1196] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/3166626c5f3c4178a386a30cfce37892
I20260620 14:53:03.649555 8890 tablet_bootstrap.cc:492] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510: Bootstrap starting.
I20260620 14:53:03.650482 8890 tablet_bootstrap.cc:654] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:03.650640 8889 tablet_bootstrap.cc:492] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37: No bootstrap required, opened a new log
I20260620 14:53:03.650707 8889 ts_tablet_manager.cc:1397] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:03.651106 8889 raft_consensus.cc:348] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.651222 8889 raft_consensus.cc:374] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:03.651268 8889 raft_consensus.cc:729] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14747d1bec724c76804fd1d56b85bb37, State: Initialized, Role: FOLLOWER
I20260620 14:53:03.651399 8889 consensus_queue.cc:260] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.651536 8911 ts_tablet_manager.cc:1939] T 3166626c5f3c4178a386a30cfce37892 P 0d4266dc82964a6c828b6ac739b93510: Deleting consensus metadata
I20260620 14:53:03.651633 8889 ts_tablet_manager.cc:1428] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:03.651922 8888 tablet_bootstrap.cc:492] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025: Bootstrap starting.
I20260620 14:53:03.652805 8369 catalog_manager.cc:4841] TS 0d4266dc82964a6c828b6ac739b93510 (127.7.24.194:34429): tablet 3166626c5f3c4178a386a30cfce37892 (table test-table [id=51624938cb7f45d79a0d7a8b2c04f06f]) successfully deleted
I20260620 14:53:03.652858 8888 tablet_bootstrap.cc:654] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:03.653854 8890 tablet_bootstrap.cc:492] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510: No bootstrap required, opened a new log
I20260620 14:53:03.653945 8890 ts_tablet_manager.cc:1397] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260620 14:53:03.654023 8888 tablet_bootstrap.cc:492] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025: No bootstrap required, opened a new log
I20260620 14:53:03.654110 8888 ts_tablet_manager.cc:1397] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:03.654441 8890 raft_consensus.cc:348] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.654596 8890 raft_consensus.cc:374] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:03.654637 8890 raft_consensus.cc:729] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d4266dc82964a6c828b6ac739b93510, State: Initialized, Role: FOLLOWER
I20260620 14:53:03.654762 8890 consensus_queue.cc:260] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.655025 8890 ts_tablet_manager.cc:1428] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 14:53:03.654969 8888 raft_consensus.cc:348] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.655069 8888 raft_consensus.cc:374] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:03.655102 8888 raft_consensus.cc:729] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48533dcce43c427ab6c6f23854fc9025, State: Initialized, Role: FOLLOWER
I20260620 14:53:03.655241 8888 consensus_queue.cc:260] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.656463 8888 ts_tablet_manager.cc:1428] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260620 14:53:03.739957 8866 tablet.cc:2369] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:53:03.777446 8896 raft_consensus.cc:482] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:03.777724 8896 raft_consensus.cc:504] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.778981 8896 leader_election.cc:288] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48533dcce43c427ab6c6f23854fc9025 (127.7.24.193:36259), 0d4266dc82964a6c828b6ac739b93510 (127.7.24.194:34429)
W20260620 14:53:03.779486 8726 tablet.cc:2369] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:53:03.785440 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.779345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:36259 (local address 127.7.24.195:33383)
0620 14:53:03.779646 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.779663 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.779763 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.780879 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.780882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.780902 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:03.781030 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.781037 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.782671 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.782674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.783955 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.783965 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.784942 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.784946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.785059 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.785083 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.785164 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.785250 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210}
I20260620 14:53:03.786190 8887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.779933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:33383 (local address 127.7.24.193:36259)
0620 14:53:03.780066 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.780070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.780084 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:03.780136 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.780139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.780181 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.780247 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:03.781261 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.782037 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.784107 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.784697 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.784825 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.784880 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.785995 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.786077 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 14:53:03.786733 8539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d26879ba8f4cfb9f520aa6d9bdcb99" candidate_uuid: "14747d1bec724c76804fd1d56b85bb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48533dcce43c427ab6c6f23854fc9025" is_pre_election: true
I20260620 14:53:03.786885 8539 raft_consensus.cc:2454] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14747d1bec724c76804fd1d56b85bb37 in term 0.
I20260620 14:53:03.787431 8750 leader_election.cc:302] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14747d1bec724c76804fd1d56b85bb37, 48533dcce43c427ab6c6f23854fc9025; no voters:
I20260620 14:53:03.787731 8896 raft_consensus.cc:2781] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:03.787783 8896 raft_consensus.cc:482] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:03.787817 8896 raft_consensus.cc:3037] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:03.787981 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.780888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:53315 (local address 127.7.24.194:34429)
0620 14:53:03.781002 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.781007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.782449 (+ 1442us) server_negotiation.cc:407] Connection header received
0620 14:53:03.782660 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.782664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.782722 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.782929 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:03.784465 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.785198 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.786714 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.787322 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.787414 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.787466 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.787792 (+ 326us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.787864 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:53:03.788349 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.780541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:34429 (local address 127.7.24.195:53315)
0620 14:53:03.782325 (+ 1784us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.782340 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.782448 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.783072 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.783076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.783099 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:03.783239 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.783246 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.785354 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.785357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.786578 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.786589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.787558 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.787562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.787663 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.787683 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.788118 (+ 435us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.788195 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1714,"thread_start_us":100,"threads_started":1}
I20260620 14:53:03.788834 8677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d26879ba8f4cfb9f520aa6d9bdcb99" candidate_uuid: "14747d1bec724c76804fd1d56b85bb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d4266dc82964a6c828b6ac739b93510" is_pre_election: true
I20260620 14:53:03.789069 8677 raft_consensus.cc:2454] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14747d1bec724c76804fd1d56b85bb37 in term 0.
I20260620 14:53:03.789065 8896 raft_consensus.cc:504] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.789474 8896 leader_election.cc:288] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [CANDIDATE]: Term 1 election: Requested vote from peers 48533dcce43c427ab6c6f23854fc9025 (127.7.24.193:36259), 0d4266dc82964a6c828b6ac739b93510 (127.7.24.194:34429)
I20260620 14:53:03.789939 8539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d26879ba8f4cfb9f520aa6d9bdcb99" candidate_uuid: "14747d1bec724c76804fd1d56b85bb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48533dcce43c427ab6c6f23854fc9025"
I20260620 14:53:03.789960 8677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d26879ba8f4cfb9f520aa6d9bdcb99" candidate_uuid: "14747d1bec724c76804fd1d56b85bb37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d4266dc82964a6c828b6ac739b93510"
I20260620 14:53:03.790064 8539 raft_consensus.cc:3037] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:03.790071 8677 raft_consensus.cc:3037] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:03.791424 8539 raft_consensus.cc:2454] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14747d1bec724c76804fd1d56b85bb37 in term 1.
I20260620 14:53:03.791777 8750 leader_election.cc:302] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14747d1bec724c76804fd1d56b85bb37, 48533dcce43c427ab6c6f23854fc9025; no voters:
I20260620 14:53:03.791869 8677 raft_consensus.cc:2454] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14747d1bec724c76804fd1d56b85bb37 in term 1.
I20260620 14:53:03.792033 8896 raft_consensus.cc:2781] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:03.792328 8896 raft_consensus.cc:686] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 1 LEADER]: Becoming Leader. State: Replica: 14747d1bec724c76804fd1d56b85bb37, State: Running, Role: LEADER
I20260620 14:53:03.792658 8896 consensus_queue.cc:237] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.795197 8382 catalog_manager.cc:5495] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14747d1bec724c76804fd1d56b85bb37 (127.7.24.195). New cstate: current_term: 1 leader_uuid: "14747d1bec724c76804fd1d56b85bb37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } health_report { overall_health: HEALTHY } } }
W20260620 14:53:03.812559 8586 tablet.cc:2369] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 14:53:03 dist-test-slave-sms9 krb5kdc[8337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967182, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:03.841004 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.832488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58726 (local address 127.7.24.254:40735)
0620 14:53:03.832649 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:03.832653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:03.832809 (+ 156us) server_negotiation.cc:407] Connection header received
0620 14:53:03.832935 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:03.832938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:03.832978 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:03.833048 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:03.833771 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.834307 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.835201 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.835443 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.835527 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.837818 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:03.837821 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:03.837823 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:03.837844 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:03.839331 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.839802 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.839806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.839808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.839888 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:03.840142 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:03.840145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:03.840146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:03.840348 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:03.840425 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:03.840789 (+ 364us) server_negotiation.cc:299] Negotiation successful
0620 14:53:03.840889 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 14:53:03.841008 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:03.832328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40735 (local address 127.0.0.1:58726)
0620 14:53:03.832715 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:03.832727 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:03.832838 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:03.833097 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:03.833099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:03.833525 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:03.833627 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.833633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.834429 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.834431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.835082 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:03.835092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:03.835553 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:03.835556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:03.835617 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:03.836187 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:03.836199 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:03.837678 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:03.839441 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.839450 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.839456 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.839701 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.839976 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:03.839978 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:03.839979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:03.840062 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:03.840473 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:03.840479 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:03.840591 (+ 112us) client_negotiation.cc:769] Sending connection context
0620 14:53:03.840697 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 14:53:03.840820 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":140,"threads_started":1}
I20260620 14:53:03.845091 8382 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:58726:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 14:53:03.845803 8382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:03.850927 8657 tablet_service.cc:1467] Processing CreateTablet for tablet 69b5d2c8b06046c3ae5c3fc1652485ac (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1db333d2278a441eb29d5b557ff380bb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 14:53:03.850934 8519 tablet_service.cc:1467] Processing CreateTablet for tablet 69b5d2c8b06046c3ae5c3fc1652485ac (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1db333d2278a441eb29d5b557ff380bb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 14:53:03.851042 8799 tablet_service.cc:1467] Processing CreateTablet for tablet 69b5d2c8b06046c3ae5c3fc1652485ac (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1db333d2278a441eb29d5b557ff380bb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 14:53:03.851332 8657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69b5d2c8b06046c3ae5c3fc1652485ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:03.851332 8519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69b5d2c8b06046c3ae5c3fc1652485ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:03.851394 8799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69b5d2c8b06046c3ae5c3fc1652485ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:03.853727 8889 tablet_bootstrap.cc:492] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37: Bootstrap starting.
I20260620 14:53:03.854084 8890 tablet_bootstrap.cc:492] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: Bootstrap starting.
I20260620 14:53:03.854635 8888 tablet_bootstrap.cc:492] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025: Bootstrap starting.
I20260620 14:53:03.854779 8889 tablet_bootstrap.cc:654] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:03.854982 8890 tablet_bootstrap.cc:654] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:03.855455 8888 tablet_bootstrap.cc:654] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:03.856179 8890 tablet_bootstrap.cc:492] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: No bootstrap required, opened a new log
I20260620 14:53:03.856280 8890 ts_tablet_manager.cc:1397] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 14:53:03.856369 8889 tablet_bootstrap.cc:492] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37: No bootstrap required, opened a new log
I20260620 14:53:03.856442 8889 ts_tablet_manager.cc:1397] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:03.856770 8888 tablet_bootstrap.cc:492] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025: No bootstrap required, opened a new log
I20260620 14:53:03.856698 8890 raft_consensus.cc:348] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.856842 8890 raft_consensus.cc:374] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:03.856878 8888 ts_tablet_manager.cc:1397] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 14:53:03.856889 8890 raft_consensus.cc:729] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d4266dc82964a6c828b6ac739b93510, State: Initialized, Role: FOLLOWER
I20260620 14:53:03.856871 8889 raft_consensus.cc:348] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.856992 8889 raft_consensus.cc:374] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:03.857023 8889 raft_consensus.cc:729] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14747d1bec724c76804fd1d56b85bb37, State: Initialized, Role: FOLLOWER
I20260620 14:53:03.857030 8890 consensus_queue.cc:260] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.857144 8889 consensus_queue.cc:260] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.857312 8890 ts_tablet_manager.cc:1428] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 14:53:03.857419 8889 ts_tablet_manager.cc:1428] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:03.857348 8888 raft_consensus.cc:348] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.857482 8888 raft_consensus.cc:374] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:03.857514 8888 raft_consensus.cc:729] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48533dcce43c427ab6c6f23854fc9025, State: Initialized, Role: FOLLOWER
I20260620 14:53:03.857542 8908 tablet_replica.cc:439] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } }
I20260620 14:53:03.857584 8896 tablet_replica.cc:439] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } }
I20260620 14:53:03.857625 8888 consensus_queue.cc:260] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:03.857674 8896 tablet_replica.cc:442] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:03.857674 8908 tablet_replica.cc:442] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:03.857861 8888 ts_tablet_manager.cc:1428] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:03.858114 8894 tablet_replica.cc:439] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } }
I20260620 14:53:03.858260 8894 tablet_replica.cc:442] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:04.174250 8908 raft_consensus.cc:482] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:04.174407 8908 raft_consensus.cc:504] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:04.174983 8908 leader_election.cc:288] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48533dcce43c427ab6c6f23854fc9025 (127.7.24.193:36259), 14747d1bec724c76804fd1d56b85bb37 (127.7.24.195:41139)
I20260620 14:53:04.175469 8819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69b5d2c8b06046c3ae5c3fc1652485ac" candidate_uuid: "0d4266dc82964a6c828b6ac739b93510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14747d1bec724c76804fd1d56b85bb37" is_pre_election: true
I20260620 14:53:04.175467 8539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69b5d2c8b06046c3ae5c3fc1652485ac" candidate_uuid: "0d4266dc82964a6c828b6ac739b93510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48533dcce43c427ab6c6f23854fc9025" is_pre_election: true
I20260620 14:53:04.175611 8819 raft_consensus.cc:2454] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d4266dc82964a6c828b6ac739b93510 in term 0.
I20260620 14:53:04.175642 8539 raft_consensus.cc:2454] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d4266dc82964a6c828b6ac739b93510 in term 0.
I20260620 14:53:04.176041 8610 leader_election.cc:302] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d4266dc82964a6c828b6ac739b93510, 48533dcce43c427ab6c6f23854fc9025; no voters:
I20260620 14:53:04.176302 8908 raft_consensus.cc:2781] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:04.176381 8908 raft_consensus.cc:482] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:04.176441 8908 raft_consensus.cc:3037] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:04.177481 8908 raft_consensus.cc:504] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:04.177847 8908 leader_election.cc:288] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [CANDIDATE]: Term 1 election: Requested vote from peers 48533dcce43c427ab6c6f23854fc9025 (127.7.24.193:36259), 14747d1bec724c76804fd1d56b85bb37 (127.7.24.195:41139)
I20260620 14:53:04.178309 8539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69b5d2c8b06046c3ae5c3fc1652485ac" candidate_uuid: "0d4266dc82964a6c828b6ac739b93510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48533dcce43c427ab6c6f23854fc9025"
I20260620 14:53:04.178313 8819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69b5d2c8b06046c3ae5c3fc1652485ac" candidate_uuid: "0d4266dc82964a6c828b6ac739b93510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14747d1bec724c76804fd1d56b85bb37"
I20260620 14:53:04.178447 8819 raft_consensus.cc:3037] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:04.178447 8539 raft_consensus.cc:3037] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:04.179625 8819 raft_consensus.cc:2454] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d4266dc82964a6c828b6ac739b93510 in term 1.
I20260620 14:53:04.179625 8539 raft_consensus.cc:2454] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d4266dc82964a6c828b6ac739b93510 in term 1.
I20260620 14:53:04.179937 8610 leader_election.cc:302] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d4266dc82964a6c828b6ac739b93510, 48533dcce43c427ab6c6f23854fc9025; no voters:
I20260620 14:53:04.180135 8908 raft_consensus.cc:2781] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:04.180332 8908 raft_consensus.cc:686] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [term 1 LEADER]: Becoming Leader. State: Replica: 0d4266dc82964a6c828b6ac739b93510, State: Running, Role: LEADER
I20260620 14:53:04.180503 8908 consensus_queue.cc:237] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } }
I20260620 14:53:04.181238 8929 tablet_replica.cc:439] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: TxnStatusTablet state changed. Reason: New leader 0d4266dc82964a6c828b6ac739b93510. Latest consensus state: current_term: 1 leader_uuid: "0d4266dc82964a6c828b6ac739b93510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } }
I20260620 14:53:04.181388 8929 tablet_replica.cc:442] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: This TxnStatusTablet replica's current role is: LEADER
I20260620 14:53:04.181607 8931 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 14:53:04.181785 8931 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 14:53:04.182404 8382 catalog_manager.cc:5495] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d4266dc82964a6c828b6ac739b93510 (127.7.24.194). New cstate: current_term: 1 leader_uuid: "0d4266dc82964a6c828b6ac739b93510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } health_report { overall_health: UNKNOWN } } }
I20260620 14:53:04.227066 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:04.223246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:34429 (local address 127.0.0.1:58856)
0620 14:53:04.223444 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:04.223462 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:04.223581 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:04.223943 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:04.223946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:04.223956 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 14:53:04.224066 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:04.224071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.224815 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.224819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:04.225453 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:04.225462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.226160 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.226164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:04.226272 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:04.226318 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 14:53:04.226744 (+ 426us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 14:53:04.226762 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 14:53:04.226851 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 14:53:04.226950 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260620 14:53:04.227063 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:04.223370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.7.24.194:34429)
0620 14:53:04.223502 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:04.223507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:04.223521 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:04.223770 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:04.223773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:04.223824 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:04.223920 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:04.224175 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.224693 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.225575 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.225812 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.226168 (+ 356us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:04.226531 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:04.226649 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:04.226737 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:04.226863 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 14:53:04.226917 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 14:53:04.231197 8539 raft_consensus.cc:1261] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025 [term 1 FOLLOWER]: Refusing update from remote peer 0d4266dc82964a6c828b6ac739b93510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:04.231182 8819 raft_consensus.cc:1261] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37 [term 1 FOLLOWER]: Refusing update from remote peer 0d4266dc82964a6c828b6ac739b93510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:04.231696 8908 consensus_queue.cc:1035] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:04.231900 8929 consensus_queue.cc:1035] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:04.233392 8896 tablet_replica.cc:439] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37: TxnStatusTablet state changed. Reason: New leader 0d4266dc82964a6c828b6ac739b93510. Latest consensus state: current_term: 1 leader_uuid: "0d4266dc82964a6c828b6ac739b93510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } }
I20260620 14:53:04.233445 8894 tablet_replica.cc:439] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025: TxnStatusTablet state changed. Reason: New leader 0d4266dc82964a6c828b6ac739b93510. Latest consensus state: current_term: 1 leader_uuid: "0d4266dc82964a6c828b6ac739b93510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } }
I20260620 14:53:04.233538 8896 tablet_replica.cc:442] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:04.233552 8894 tablet_replica.cc:442] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:04.240011 8908 tablet_replica.cc:439] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d4266dc82964a6c828b6ac739b93510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } }
I20260620 14:53:04.240132 8908 tablet_replica.cc:442] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: This TxnStatusTablet replica's current role is: LEADER
I20260620 14:53:04.240432 8896 tablet_replica.cc:439] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d4266dc82964a6c828b6ac739b93510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } }
I20260620 14:53:04.240540 8896 tablet_replica.cc:442] T 69b5d2c8b06046c3ae5c3fc1652485ac P 14747d1bec724c76804fd1d56b85bb37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:04.241106 8929 tablet_replica.cc:439] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d4266dc82964a6c828b6ac739b93510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } }
I20260620 14:53:04.241218 8929 tablet_replica.cc:442] T 69b5d2c8b06046c3ae5c3fc1652485ac P 0d4266dc82964a6c828b6ac739b93510: This TxnStatusTablet replica's current role is: LEADER
I20260620 14:53:04.242653 8894 tablet_replica.cc:439] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d4266dc82964a6c828b6ac739b93510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 } } peers { permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 } } peers { permanent_uuid: "14747d1bec724c76804fd1d56b85bb37" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41139 } } }
I20260620 14:53:04.242791 8894 tablet_replica.cc:442] T 69b5d2c8b06046c3ae5c3fc1652485ac P 48533dcce43c427ab6c6f23854fc9025: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:04.252348 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:04.248408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54646 (local address 127.7.24.195:41139)
0620 14:53:04.248634 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:04.248640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:04.248698 (+ 58us) server_negotiation.cc:407] Connection header received
0620 14:53:04.248899 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:04.248902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:04.248971 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:04.249101 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:04.249347 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.249904 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.250821 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.251066 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.251178 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:04.251454 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:04.251934 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:04.252063 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:04.252127 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 14:53:04.252179 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":81,"threads_started":1}
I20260620 14:53:04.262737 8947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:04.259315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:34429 (local address 127.7.24.195:55027)
0620 14:53:04.259696 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:04.259708 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:04.259822 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:04.260079 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:04.260081 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:04.260089 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 14:53:04.260198 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:04.260206 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.260901 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.260908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:04.261539 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:04.261547 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.261988 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.261991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:04.262088 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:04.262126 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 14:53:04.262423 (+ 297us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 14:53:04.262429 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 14:53:04.262530 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 14:53:04.262612 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":123,"threads_started":1}
I20260620 14:53:04.262729 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:04.259421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:55027 (local address 127.7.24.194:34429)
0620 14:53:04.259557 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:04.259564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:04.259776 (+ 212us) server_negotiation.cc:407] Connection header received
0620 14:53:04.259941 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:04.259944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:04.259986 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:04.260069 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:04.260307 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.260791 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.261658 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.261875 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.261976 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:04.262268 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:04.262332 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:04.262407 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:04.262541 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 14:53:04.262590 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 14:53:04.269673 8539 raft_consensus.cc:1261] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025 [term 1 FOLLOWER]: Refusing update from remote peer 14747d1bec724c76804fd1d56b85bb37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:04.269865 8677 raft_consensus.cc:1261] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510 [term 1 FOLLOWER]: Refusing update from remote peer 14747d1bec724c76804fd1d56b85bb37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:04.270144 8896 consensus_queue.cc:1035] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48533dcce43c427ab6c6f23854fc9025" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:04.270340 8918 consensus_queue.cc:1035] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d4266dc82964a6c828b6ac739b93510" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:04.302778 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:04.289596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:52593 (local address 127.7.24.195:41139)
0620 14:53:04.289707 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:04.289712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:04.298727 (+ 9015us) server_negotiation.cc:407] Connection header received
0620 14:53:04.298912 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:04.298916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:04.298969 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:04.299015 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:04.299344 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.300134 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.301329 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.301688 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.301804 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:04.302222 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:04.302311 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:04.302386 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:04.302577 (+ 191us) server_negotiation.cc:299] Negotiation successful
0620 14:53:04.302638 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 14:53:04.303158 8954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:04.289555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:41139 (local address 127.7.24.194:52593)
0620 14:53:04.298645 (+ 9090us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:04.298662 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:04.298798 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:04.299144 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:04.299147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:04.299154 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 14:53:04.299249 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:04.299254 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.300264 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.300267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:04.301194 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:04.301203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.301814 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.301817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:04.301902 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:04.301956 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 14:53:04.302440 (+ 484us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 14:53:04.302447 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 14:53:04.302938 (+ 491us) client_negotiation.cc:240] Negotiation successful
0620 14:53:04.303023 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8964,"thread_start_us":86,"threads_started":1}
I20260620 14:53:04.326973 8380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:04.327157 8380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:04.329193 8380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e28baffb5a34549a896555398674ee8: Sending DeleteTablet for 3 replicas of tablet 54d26879ba8f4cfb9f520aa6d9bdcb99
I20260620 14:53:04.329780 8519 tablet_service.cc:1514] Processing DeleteTablet for tablet 54d26879ba8f4cfb9f520aa6d9bdcb99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:04 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:40556
I20260620 14:53:04.329888 8799 tablet_service.cc:1514] Processing DeleteTablet for tablet 54d26879ba8f4cfb9f520aa6d9bdcb99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:04 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54640
I20260620 14:53:04.330507 8657 tablet_service.cc:1514] Processing DeleteTablet for tablet 54d26879ba8f4cfb9f520aa6d9bdcb99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:04 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:58840
I20260620 14:53:04.330920 8959 tablet_replica.cc:330] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510: stopping tablet replica
I20260620 14:53:04.331089 8959 raft_consensus.cc:2229] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:04.331272 8959 raft_consensus.cc:2258] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:04.332190 8959 ts_tablet_manager.cc:1905] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:04.335662 8959 ts_tablet_manager.cc:1918] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:04.335880 8959 log.cc:1196] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/54d26879ba8f4cfb9f520aa6d9bdcb99
I20260620 14:53:04.336244 8959 ts_tablet_manager.cc:1939] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 0d4266dc82964a6c828b6ac739b93510: Deleting consensus metadata
I20260620 14:53:04.337157 8369 catalog_manager.cc:4841] TS 0d4266dc82964a6c828b6ac739b93510 (127.7.24.194:34429): tablet 54d26879ba8f4cfb9f520aa6d9bdcb99 (table test-table [id=1782d9c7ac964402954faa86c6b6178f]) successfully deleted
I20260620 14:53:04.338635 8957 tablet_replica.cc:330] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025: stopping tablet replica
I20260620 14:53:04.338794 8957 raft_consensus.cc:2229] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:04.338937 8957 raft_consensus.cc:2258] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:04.339196 8958 tablet_replica.cc:330] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37: stopping tablet replica
I20260620 14:53:04.339320 8958 raft_consensus.cc:2229] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:04.339641 8958 raft_consensus.cc:2258] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37 [term 1 FOLLOWER]: Raft consensus is shut down!
Jun 20 14:53:04 dist-test-slave-sms9 krb5kdc[8337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967183, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:53:04.343042 8957 ts_tablet_manager.cc:1905] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:04.344873 8958 ts_tablet_manager.cc:1905] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:04.345443 8961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:04.332170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.7.24.193:36259)
0620 14:53:04.332790 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:04.332799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:04.333220 (+ 421us) server_negotiation.cc:407] Connection header received
0620 14:53:04.333265 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:04.333268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:04.333316 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:04.333405 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:04.334431 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.335317 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.336694 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.337058 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.337157 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:04.340973 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:04.340987 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:04.341005 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:04.341040 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:04.343051 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:04.343652 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:04.343659 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:04.343660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:04.343749 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:04.344051 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:04.344055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:04.344056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:04.344339 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:04.344549 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:04.345103 (+ 554us) server_negotiation.cc:299] Negotiation successful
0620 14:53:04.345229 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":111,"threads_started":1}
W20260620 14:53:04.346506 8579 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40566
I20260620 14:53:04.349453 8957 ts_tablet_manager.cc:1918] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:04.349552 8957 log.cc:1196] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-0/wal/wals/54d26879ba8f4cfb9f520aa6d9bdcb99
I20260620 14:53:04.349852 8957 ts_tablet_manager.cc:1939] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 48533dcce43c427ab6c6f23854fc9025: Deleting consensus metadata
I20260620 14:53:04.350632 8366 catalog_manager.cc:4841] TS 48533dcce43c427ab6c6f23854fc9025 (127.7.24.193:36259): tablet 54d26879ba8f4cfb9f520aa6d9bdcb99 (table test-table [id=1782d9c7ac964402954faa86c6b6178f]) successfully deleted
I20260620 14:53:04.352015 8958 ts_tablet_manager.cc:1918] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:04.352115 8958 log.cc:1196] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/54d26879ba8f4cfb9f520aa6d9bdcb99
I20260620 14:53:04.352370 8958 ts_tablet_manager.cc:1939] T 54d26879ba8f4cfb9f520aa6d9bdcb99 P 14747d1bec724c76804fd1d56b85bb37: Deleting consensus metadata
I20260620 14:53:04.353134 8367 catalog_manager.cc:4841] TS 14747d1bec724c76804fd1d56b85bb37 (127.7.24.195:41139): tablet 54d26879ba8f4cfb9f520aa6d9bdcb99 (table test-table [id=1782d9c7ac964402954faa86c6b6178f]) successfully deleted
I20260620 14:53:04.357143 8965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:04.348578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58738 (local address 127.7.24.254:40735)
0620 14:53:04.348795 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:04.348799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:04.348815 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:53:04.348910 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:04.348913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:04.348961 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:04.349052 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:04.349812 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.350626 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.351686 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:04.352058 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:04.352158 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:04.353870 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:04.353874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:04.353876 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:04.353902 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:04.355466 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:04.355937 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:04.355940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:04.355942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:04.356024 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:04.356287 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:04.356289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:04.356290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:04.356527 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:04.356628 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:04.356868 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 14:53:04.356967 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":60,"threads_started":1}
W20260620 14:53:04.357527 8380 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58738
I20260620 14:53:04.359042 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 8454
I20260620 14:53:04.366895 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 8594
I20260620 14:53:04.374643 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 8734
I20260620 14:53:04.379772 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 8350
2026-06-20T14:53:04Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3014 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:04 dist-test-slave-sms9 krb5kdc[8968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:04 dist-test-slave-sms9 krb5kdc[8968](info): set up 2 sockets
Jun 20 14:53:04 dist-test-slave-sms9 krb5kdc[8968](info): commencing operation
krb5kdc: starting...
W20260620 14:53:05.450399 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:05 dist-test-slave-sms9 krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:05.577445 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:33969
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:33969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:05.686710 8981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:05.686991 8981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:05.687033 8981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:05.691072 8981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:05.691133 8981 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 14:53:05.691150 8981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:05.691169 8981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:05.691185 8981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:05.691239 8981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:05.696631 8981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:33969
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:33969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:05.697767 8981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:05.698832 8981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:05.707472 8989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:05.707472 8986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:05.707512 8987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:05.707743 8981 server_base.cc:1034] running on GCE node
I20260620 14:53:05.708438 8981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:05.709700 8981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:05.710901 8981 hybrid_clock.cc:648] HybridClock initialized: now 1781967185710884 us; error 37 us; skew 500 ppm
Jun 20 14:53:05 dist-test-slave-sms9 krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967185, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:05.713744 8981 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:05.714754 8981 webserver.cc:460] Webserver started at http://127.7.24.254:45775/ using document root <none> and password file <none>
I20260620 14:53:05.715332 8981 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:05.715445 8981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:05.715682 8981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:05.717727 8981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "4245a551ded442dfb9405268d5f4fd33"
format_stamp: "Formatted at 2026-06-20 14:53:05 on dist-test-slave-sms9"
server_key: "b4e266be8976d56c8b2593e1447832d3"
server_key_iv: "a15b99c3488d90bbfb2f6516a19555b2"
server_key_version: "encryptionkey@0"
I20260620 14:53:05.718313 8981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "4245a551ded442dfb9405268d5f4fd33"
format_stamp: "Formatted at 2026-06-20 14:53:05 on dist-test-slave-sms9"
server_key: "b4e266be8976d56c8b2593e1447832d3"
server_key_iv: "a15b99c3488d90bbfb2f6516a19555b2"
server_key_version: "encryptionkey@0"
I20260620 14:53:05.722081 8981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 14:53:05.724663 8996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:05.726027 8981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:05.726182 8981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "4245a551ded442dfb9405268d5f4fd33"
format_stamp: "Formatted at 2026-06-20 14:53:05 on dist-test-slave-sms9"
server_key: "b4e266be8976d56c8b2593e1447832d3"
server_key_iv: "a15b99c3488d90bbfb2f6516a19555b2"
server_key_version: "encryptionkey@0"
I20260620 14:53:05.726310 8981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:05.740309 8981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:05.743666 8981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:05.743878 8981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:05.751979 8981 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:33969
I20260620 14:53:05.752015 9058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:33969 every 8 connection(s)
I20260620 14:53:05.752967 8981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:05.753733 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 8981
I20260620 14:53:05.753867 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:05.754163 7267 external_mini_cluster.cc:1428] Setting key 9ec84c94a35cff46a10fb9cb6e5218f9
I20260620 14:53:05.756678 9059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:05.762706 9059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33: Bootstrap starting.
Jun 20 14:53:05 dist-test-slave-sms9 krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:05.765743 9059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:05.767325 9059 log.cc:824] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:05.769835 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:05.755757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55482 (local address 127.7.24.254:33969)
0620 14:53:05.756222 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:05.756237 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:05.756273 (+ 36us) server_negotiation.cc:407] Connection header received
0620 14:53:05.757012 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:05.757036 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:05.757366 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:05.757708 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:05.758582 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:05.759623 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:05.760509 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:05.761002 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:05.761170 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:05.763831 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:05.763838 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:05.763858 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:05.763902 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:05.766205 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:05.766806 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:05.766812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:05.766819 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:05.766970 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:05.767291 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:05.767295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:05.767297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:05.767868 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:05.768027 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:05.768510 (+ 483us) server_negotiation.cc:299] Negotiation successful
0620 14:53:05.768831 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":139,"threads_started":1}
I20260620 14:53:05.769958 9059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33: No bootstrap required, opened a new log
I20260620 14:53:05.773644 9059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4245a551ded442dfb9405268d5f4fd33" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33969 } }
I20260620 14:53:05.773866 9059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:05.773901 9059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4245a551ded442dfb9405268d5f4fd33, State: Initialized, Role: FOLLOWER
I20260620 14:53:05.774240 9059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4245a551ded442dfb9405268d5f4fd33" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33969 } }
I20260620 14:53:05.774350 9059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:05.774387 9059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:05.774505 9059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:05.775377 9059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4245a551ded442dfb9405268d5f4fd33" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33969 } }
I20260620 14:53:05.775668 9059 leader_election.cc:302] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4245a551ded442dfb9405268d5f4fd33; no voters:
I20260620 14:53:05.775946 9059 leader_election.cc:288] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:05.776142 9064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:05.776727 9064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [term 1 LEADER]: Becoming Leader. State: Replica: 4245a551ded442dfb9405268d5f4fd33, State: Running, Role: LEADER
I20260620 14:53:05.777109 9064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4245a551ded442dfb9405268d5f4fd33" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33969 } }
I20260620 14:53:05.777259 9059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:05.778774 9066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4245a551ded442dfb9405268d5f4fd33. Latest consensus state: current_term: 1 leader_uuid: "4245a551ded442dfb9405268d5f4fd33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4245a551ded442dfb9405268d5f4fd33" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33969 } } }
I20260620 14:53:05.778824 9065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4245a551ded442dfb9405268d5f4fd33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4245a551ded442dfb9405268d5f4fd33" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33969 } } }
I20260620 14:53:05.778923 9065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:05.778923 9066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33 [sys.catalog]: This master's current role is: LEADER
W20260620 14:53:05.782372 9079 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:53:05.782524 9079 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:53:05.782626 9076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:05.783382 9076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:05.789223 9076 catalog_manager.cc:1347] Generated new cluster ID: 8dcfc4b54c394b24a210e7ecff4316e0
I20260620 14:53:05.789314 9076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:05.801018 9076 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:05.802330 9076 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:05.808197 9076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33: Generated new TSK 0
I20260620 14:53:05.809053 9076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:53:05.841971 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:33969
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:05.955981 9085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:05.956279 9085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:05.956331 9085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:05.960594 9085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:05.960670 9085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:05.960714 9085 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 14:53:05.960728 9085 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 14:53:05.960806 9085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:05.966391 9085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.24.254:33969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:05.967629 9085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:05.968739 9085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:05.978816 9090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:05.978946 9093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:05.979117 9085 server_base.cc:1034] running on GCE node
W20260620 14:53:05.979323 9091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:05.979768 9085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:05.980466 9085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:05.981673 9085 hybrid_clock.cc:648] HybridClock initialized: now 1781967185981634 us; error 54 us; skew 500 ppm
Jun 20 14:53:05 dist-test-slave-sms9 krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967185, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:05.984768 9085 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:53:05.985714 9085 webserver.cc:460] Webserver started at http://127.7.24.193:43013/ using document root <none> and password file <none>
I20260620 14:53:05.986315 9085 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:05.986409 9085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:05.986665 9085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:05.989010 9085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "274bb23d20a44efe95b8fdca492306b8"
format_stamp: "Formatted at 2026-06-20 14:53:05 on dist-test-slave-sms9"
server_key: "6ef634c8e78d08cd0fd26d97e86fec9f"
server_key_iv: "4f798b3496da74d3597a42d58b4a3050"
server_key_version: "encryptionkey@0"
I20260620 14:53:05.989640 9085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "274bb23d20a44efe95b8fdca492306b8"
format_stamp: "Formatted at 2026-06-20 14:53:05 on dist-test-slave-sms9"
server_key: "6ef634c8e78d08cd0fd26d97e86fec9f"
server_key_iv: "4f798b3496da74d3597a42d58b4a3050"
server_key_version: "encryptionkey@0"
I20260620 14:53:05.993592 9085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:53:05.996423 9100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:05.997890 9085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:05.998056 9085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "274bb23d20a44efe95b8fdca492306b8"
format_stamp: "Formatted at 2026-06-20 14:53:05 on dist-test-slave-sms9"
server_key: "6ef634c8e78d08cd0fd26d97e86fec9f"
server_key_iv: "4f798b3496da74d3597a42d58b4a3050"
server_key_version: "encryptionkey@0"
I20260620 14:53:05.998189 9085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:06.022619 9085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:06.026171 9085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:06.026337 9085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:06.027954 9085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:06.028005 9085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:06.028048 9085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:06.028062 9085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 14:53:06 dist-test-slave-sms9 krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967185, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:06.039564 9085 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:33381
I20260620 14:53:06.039598 9215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:33381 every 8 connection(s)
I20260620 14:53:06.040624 9085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:06.043912 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.028975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:56921 (local address 127.7.24.254:33969)
0620 14:53:06.029121 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.029125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.030498 (+ 1373us) server_negotiation.cc:407] Connection header received
0620 14:53:06.031733 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.031740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.031810 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.031924 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:06.033315 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.033948 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.035525 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.035853 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.035991 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.039877 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:06.039881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:06.039884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:06.039922 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:06.041643 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.042167 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.042170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.042172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.042263 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.042692 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.042695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.042697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.042896 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:06.043015 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.043585 (+ 570us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.043726 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 14:53:06.044752 9109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.029555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:33969 (local address 127.7.24.193:56921)
0620 14:53:06.030271 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.030317 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.031471 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.032135 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.032142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.032557 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:06.033152 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.033166 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.034078 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.034082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.035384 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.035399 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.036049 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.036053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.036241 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.037541 (+ 1300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:06.037558 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:06.039618 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:06.041767 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.041773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.041788 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.042051 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.042370 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.042373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.042375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.042586 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.043025 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:06.043034 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:06.043294 (+ 260us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.043531 (+ 237us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.043869 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":153,"threads_started":1}
I20260620 14:53:06.046187 9216 heartbeater.cc:347] Connected to a master server at 127.7.24.254:33969
I20260620 14:53:06.046360 9216 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:06.046913 9216 heartbeater.cc:511] Master 127.7.24.254:33969 requested a full tablet report, sending...
I20260620 14:53:06.048705 9012 ts_manager.cc:194] Registered new tserver with Master: 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381)
I20260620 14:53:06.049363 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 9085
I20260620 14:53:06.049531 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:06.049865 7267 external_mini_cluster.cc:1428] Setting key 44dc1ee2cda722e725f847bdc245c6b5
I20260620 14:53:06.050220 9012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:56921
I20260620 14:53:06.057097 9222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.048558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:33969 (local address 127.7.24.193:34373)
0620 14:53:06.048850 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.048867 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.048969 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.049276 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.049279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.049507 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:06.049634 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.049641 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.050657 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.050660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.051443 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.051452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.051928 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.051931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.052057 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.052636 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:06.052648 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:06.053093 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:06.055195 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.055198 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.055199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.055432 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.055764 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.055766 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.055767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.055974 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.056502 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:06.056505 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:06.056599 (+ 94us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.056765 (+ 166us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.056907 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":20,"thread_start_us":98,"threads_started":1}
I20260620 14:53:06.057230 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.048634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:34373 (local address 127.7.24.254:33969)
0620 14:53:06.048778 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.048782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.048923 (+ 141us) server_negotiation.cc:407] Connection header received
0620 14:53:06.049068 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.049070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.049143 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.049233 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:06.049759 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.050526 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.051570 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.051808 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.051928 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.053239 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:06.053242 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:06.053244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:06.053266 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:06.055061 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.055540 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.055547 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.055552 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.055648 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.056074 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.056080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.056085 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.056367 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:06.056491 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.056840 (+ 349us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.056996 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:53:06.078053 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:33969
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:06.189410 9225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:06.189669 9225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:06.189716 9225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:06.193744 9225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:06.193809 9225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:06.193853 9225 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 14:53:06.193869 9225 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 14:53:06.193902 9225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:06.199261 9225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.24.254:33969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:06.200299 9225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:06.201356 9225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:06.212008 9231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:06.212029 9230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:06.212029 9233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:06.212361 9225 server_base.cc:1034] running on GCE node
I20260620 14:53:06.212807 9225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:06.213505 9225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:06.214838 9225 hybrid_clock.cc:648] HybridClock initialized: now 1781967186214682 us; error 50 us; skew 500 ppm
Jun 20 14:53:06 dist-test-slave-sms9 krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967186, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:06.217990 9225 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:53:06.218974 9225 webserver.cc:460] Webserver started at http://127.7.24.194:46143/ using document root <none> and password file <none>
I20260620 14:53:06.219524 9225 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:06.219584 9225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:06.219764 9225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:06.221921 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "52fe6429041c498db60532f938ea38e7"
format_stamp: "Formatted at 2026-06-20 14:53:06 on dist-test-slave-sms9"
server_key: "b7e49ab296e32e2824ae29b757935f70"
server_key_iv: "1f5c2b2d3cf400b99735bdcd8c2e5a48"
server_key_version: "encryptionkey@0"
I20260620 14:53:06.222438 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "52fe6429041c498db60532f938ea38e7"
format_stamp: "Formatted at 2026-06-20 14:53:06 on dist-test-slave-sms9"
server_key: "b7e49ab296e32e2824ae29b757935f70"
server_key_iv: "1f5c2b2d3cf400b99735bdcd8c2e5a48"
server_key_version: "encryptionkey@0"
I20260620 14:53:06.226189 9225 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:06.228714 9240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:06.230120 9225 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:06.230252 9225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "52fe6429041c498db60532f938ea38e7"
format_stamp: "Formatted at 2026-06-20 14:53:06 on dist-test-slave-sms9"
server_key: "b7e49ab296e32e2824ae29b757935f70"
server_key_iv: "1f5c2b2d3cf400b99735bdcd8c2e5a48"
server_key_version: "encryptionkey@0"
I20260620 14:53:06.230403 9225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:06.248150 9225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:06.251606 9225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:06.251827 9225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:06.253402 9225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:06.253453 9225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:06.253494 9225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:06.253551 9225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 14:53:06 dist-test-slave-sms9 krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967186, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:06.264897 9225 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:38547
I20260620 14:53:06.264992 9355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:38547 every 8 connection(s)
I20260620 14:53:06.265834 9225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:06.269366 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.254583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:54081 (local address 127.7.24.254:33969)
0620 14:53:06.254719 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.254723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.255959 (+ 1236us) server_negotiation.cc:407] Connection header received
0620 14:53:06.257046 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.257050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.257112 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.257199 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:06.259028 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.259641 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.261559 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.261868 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.261961 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.265051 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:06.265056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:06.265059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:06.265097 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:06.267054 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.267735 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.267738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.267739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.267826 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.268185 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.268189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.268190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.268411 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:06.268519 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.269062 (+ 543us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.269197 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:53:06.270182 9249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.255211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:33969 (local address 127.7.24.194:54081)
0620 14:53:06.255753 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.255796 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.256825 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.257488 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.257499 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.258019 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:06.258825 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.258851 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.259768 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.259772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.261383 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.261397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.262042 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.262048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.262302 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.263068 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:06.263093 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:06.264818 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:06.267189 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.267196 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.267213 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.267608 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.267922 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.267925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.267927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.268077 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.268518 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:06.268527 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:06.268817 (+ 290us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.269008 (+ 191us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.269356 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":135,"threads_started":1}
I20260620 14:53:06.271423 9356 heartbeater.cc:347] Connected to a master server at 127.7.24.254:33969
I20260620 14:53:06.271595 9356 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:06.272117 9356 heartbeater.cc:511] Master 127.7.24.254:33969 requested a full tablet report, sending...
I20260620 14:53:06.273227 9012 ts_manager.cc:194] Registered new tserver with Master: 52fe6429041c498db60532f938ea38e7 (127.7.24.194:38547)
I20260620 14:53:06.273878 9012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:54081
I20260620 14:53:06.275576 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 9225
I20260620 14:53:06.275712 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:06.276049 7267 external_mini_cluster.cc:1428] Setting key 9dceb098bcc904020e84039d7db9755a
I20260620 14:53:06.283898 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.274591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:51379 (local address 127.7.24.254:33969)
0620 14:53:06.274710 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.274715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.275169 (+ 454us) server_negotiation.cc:407] Connection header received
0620 14:53:06.275257 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.275260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.275308 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.275399 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:06.275992 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.276736 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.277915 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.278298 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.278442 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.279808 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:06.279813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:06.279817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:06.279849 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:06.281550 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.282141 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.282144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.282145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.282230 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.282620 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.282626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.282630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.282981 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:06.283068 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.283589 (+ 521us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.283715 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 14:53:06.283895 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.274735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:33969 (local address 127.7.24.194:51379)
0620 14:53:06.275071 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.275084 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.275172 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.275436 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.275439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.275707 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:06.275845 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.275853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.276885 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.276888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.277789 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.277799 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.278427 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.278431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.278577 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.279217 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:06.279229 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:06.279619 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:06.281686 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.281692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.281697 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.282003 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.282340 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.282346 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.282351 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.282488 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.283148 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:06.283151 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:06.283277 (+ 126us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.283394 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.283594 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":161,"threads_started":1}
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:53:06.306519 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:33969
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:06.431069 9365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:06.431380 9365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:06.431432 9365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:06.436173 9365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:06.436254 9365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:06.436302 9365 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 14:53:06.436318 9365 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 14:53:06.436355 9365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:06.442283 9365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.24.254:33969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:06.443576 9365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:06.444599 9365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:53:06.454916 9365 server_base.cc:1034] running on GCE node
W20260620 14:53:06.454859 9370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:06.454871 9371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:06.454859 9373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:06.455619 9365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:06.456441 9365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:06.457619 9365 hybrid_clock.cc:648] HybridClock initialized: now 1781967186457587 us; error 59 us; skew 500 ppm
Jun 20 14:53:06 dist-test-slave-sms9 krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967186, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:06.460618 9365 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:53:06.461601 9365 webserver.cc:460] Webserver started at http://127.7.24.195:41317/ using document root <none> and password file <none>
I20260620 14:53:06.462232 9365 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:06.462306 9365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:06.462618 9365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:06.465523 9365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "6c68ae7817e040729aaa92be7337ddc8"
format_stamp: "Formatted at 2026-06-20 14:53:06 on dist-test-slave-sms9"
server_key: "afefce72c00a8888da3eb8fc5737ff7e"
server_key_iv: "109686e01b668c41fdb4fb078db3b277"
server_key_version: "encryptionkey@0"
I20260620 14:53:06.466205 9365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "6c68ae7817e040729aaa92be7337ddc8"
format_stamp: "Formatted at 2026-06-20 14:53:06 on dist-test-slave-sms9"
server_key: "afefce72c00a8888da3eb8fc5737ff7e"
server_key_iv: "109686e01b668c41fdb4fb078db3b277"
server_key_version: "encryptionkey@0"
I20260620 14:53:06.470265 9365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 14:53:06.473220 9380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:06.474665 9365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:06.474813 9365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "6c68ae7817e040729aaa92be7337ddc8"
format_stamp: "Formatted at 2026-06-20 14:53:06 on dist-test-slave-sms9"
server_key: "afefce72c00a8888da3eb8fc5737ff7e"
server_key_iv: "109686e01b668c41fdb4fb078db3b277"
server_key_version: "encryptionkey@0"
I20260620 14:53:06.474983 9365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:06.504578 9365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:06.507807 9365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:06.507995 9365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:06.509479 9365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:06.509557 9365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:06.509630 9365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:06.509683 9365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 14:53:06 dist-test-slave-sms9 krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967186, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:06.520637 9365 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:43839
I20260620 14:53:06.520759 9495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:43839 every 8 connection(s)
I20260620 14:53:06.521857 9365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:06.524026 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.510270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:39153 (local address 127.7.24.254:33969)
0620 14:53:06.510446 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.510452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.511773 (+ 1321us) server_negotiation.cc:407] Connection header received
0620 14:53:06.513183 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.513186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.513259 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.513360 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:06.514747 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.515377 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.516809 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.517121 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.517238 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.520208 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:06.520212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:06.520215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:06.520245 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:06.521964 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.522525 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.522529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.522531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.522618 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.523045 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.523048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.523049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.523236 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:06.523316 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.523783 (+ 467us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.523905 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 14:53:06.524137 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 9365
I20260620 14:53:06.524242 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:06.524519 7267 external_mini_cluster.cc:1428] Setting key 85c5e458ea20a2a2f01492d67d1dd554
I20260620 14:53:06.525023 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.511123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:33969 (local address 127.7.24.195:39153)
0620 14:53:06.511610 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.511668 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.512954 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.513571 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.513578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.514030 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:06.514599 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.514616 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.515520 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.515523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.516670 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.516680 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.517276 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.517280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.517478 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.518138 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:06.518154 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:06.520032 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:06.522070 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.522075 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.522086 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.522381 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.522763 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.522766 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.522770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.522930 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.523349 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:06.523354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:06.523571 (+ 217us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.523717 (+ 146us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.524029 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":101,"threads_started":1}
I20260620 14:53:06.526171 9496 heartbeater.cc:347] Connected to a master server at 127.7.24.254:33969
I20260620 14:53:06.526324 9496 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:06.526916 9496 heartbeater.cc:511] Master 127.7.24.254:33969 requested a full tablet report, sending...
I20260620 14:53:06.527980 9012 ts_manager.cc:194] Registered new tserver with Master: 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195:43839)
I20260620 14:53:06.528561 9012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:39153
I20260620 14:53:06.536140 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.528430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:33969 (local address 127.7.24.195:47617)
0620 14:53:06.528723 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.528747 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.528859 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.529193 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.529195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.529359 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:06.529454 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.529459 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.530650 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.530653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.531273 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.531282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.531747 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.531750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.531825 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.532387 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:06.532399 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:06.532712 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:06.534598 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.534601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.534602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.534796 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.535238 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.535241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.535242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.535328 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.535669 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:06.535671 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:06.535770 (+ 99us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.535859 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.535980 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":112,"threads_started":1}
I20260620 14:53:06.536237 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.528737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:47617 (local address 127.7.24.254:33969)
0620 14:53:06.528975 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.528979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.528992 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:53:06.529032 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.529034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.529070 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.529175 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:06.529612 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.530495 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.531427 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.531637 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.531725 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.532901 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:06.532905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:06.532907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:06.532930 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:06.534403 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.535003 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.535006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.535007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.535083 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.535420 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.535422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.535423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.535588 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:06.535665 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.536024 (+ 359us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.536123 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260620 14:53:06.538551 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 14:53:06 dist-test-slave-sms9 krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 14:53:06 dist-test-slave-sms9 krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:06.565071 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.556551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55490 (local address 127.7.24.254:33969)
0620 14:53:06.556702 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.556706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.556720 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:06.556763 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.556766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.556817 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.556926 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:06.557636 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.558185 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.559110 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.559344 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.559427 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.561918 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:06.561921 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:06.561923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:06.561950 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:06.563276 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.563753 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.563755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.563757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.563841 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.564117 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.564120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.564121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.564321 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:06.564436 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.564684 (+ 248us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.564898 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 14:53:06.568320 9013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:06.571414 9013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:06.587194 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.581583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.7.24.194:38547)
0620 14:53:06.581977 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.581990 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.582209 (+ 219us) server_negotiation.cc:407] Connection header received
0620 14:53:06.582344 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.582349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.582583 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.582732 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:06.583098 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.583971 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.585580 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.586171 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.586306 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.586540 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.586944 (+ 404us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.587044 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":99,"threads_started":1}
I20260620 14:53:06.587182 9515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.580975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38547 (local address 127.0.0.1:58532)
0620 14:53:06.582110 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.582121 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.582227 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.582746 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.582752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.582778 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:06.582953 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.582963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.584226 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.584230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.585446 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.585456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.586605 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.586609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.586735 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.586775 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.586895 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.587017 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1075,"thread_start_us":103,"threads_started":1}
I20260620 14:53:06.587652 9516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.580815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:43839 (local address 127.0.0.1:33102)
0620 14:53:06.581730 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.581781 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.581910 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.582948 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.582951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.582969 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:06.583081 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.583088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.584528 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.584531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.586148 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.586158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.587231 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.587235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.587351 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.587383 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.587451 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.587512 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":181,"thread_start_us":67,"threads_started":1}
I20260620 14:53:06.588074 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.581663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58590 (local address 127.7.24.193:33381)
0620 14:53:06.582109 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.582115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.582331 (+ 216us) server_negotiation.cc:407] Connection header received
0620 14:53:06.582528 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.582533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.582734 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.582876 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:06.583296 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.584387 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.586285 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.587169 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.587298 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.587445 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.587818 (+ 373us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.587919 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":86,"threads_started":1}
I20260620 14:53:06.588088 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.580960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33102 (local address 127.7.24.195:43839)
0620 14:53:06.582494 (+ 1534us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.582509 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.582530 (+ 21us) server_negotiation.cc:407] Connection header received
0620 14:53:06.582597 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.582601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.582761 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.582960 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:06.583199 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.584403 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.586286 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.586929 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.587027 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.587167 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.587867 (+ 700us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.587958 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1413,"thread_start_us":119,"threads_started":1}
I20260620 14:53:06.588445 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.581227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33381 (local address 127.0.0.1:58590)
0620 14:53:06.582226 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.582238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.582362 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.583009 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.583015 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.583038 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:06.583134 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.583144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.584528 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.584531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.586148 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.586158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.587516 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.587520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.587621 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.587658 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.588257 (+ 599us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.588344 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"mutex_wait_us":261,"thread_start_us":109,"threads_started":1}
I20260620 14:53:06.590524 9290 tablet_service.cc:1467] Processing CreateTablet for tablet d0560c37074a4c1fa186c9d780bfe3bc (DEFAULT_TABLE table=test-table [id=b521637adcb94e208d433a79867bf356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:06.590670 9150 tablet_service.cc:1467] Processing CreateTablet for tablet d0560c37074a4c1fa186c9d780bfe3bc (DEFAULT_TABLE table=test-table [id=b521637adcb94e208d433a79867bf356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:06.591598 9290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0560c37074a4c1fa186c9d780bfe3bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:06.591699 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0560c37074a4c1fa186c9d780bfe3bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:06.596601 9418 tablet_service.cc:1467] Processing CreateTablet for tablet d0560c37074a4c1fa186c9d780bfe3bc (DEFAULT_TABLE table=test-table [id=b521637adcb94e208d433a79867bf356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:06.597947 9418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0560c37074a4c1fa186c9d780bfe3bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:06.598410 9519 tablet_bootstrap.cc:492] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: Bootstrap starting.
I20260620 14:53:06.600666 9520 tablet_bootstrap.cc:492] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: Bootstrap starting.
I20260620 14:53:06.600832 9519 tablet_bootstrap.cc:654] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:06.602105 9519 log.cc:824] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:06.603051 9520 tablet_bootstrap.cc:654] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:06.603849 9520 log.cc:824] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:06.604311 9519 tablet_bootstrap.cc:492] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: No bootstrap required, opened a new log
I20260620 14:53:06.604530 9519 ts_tablet_manager.cc:1397] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:53:06.605222 9523 tablet_bootstrap.cc:492] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: Bootstrap starting.
I20260620 14:53:06.605985 9520 tablet_bootstrap.cc:492] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: No bootstrap required, opened a new log
I20260620 14:53:06.606200 9520 ts_tablet_manager.cc:1397] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:53:06.607172 9523 tablet_bootstrap.cc:654] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:06.607818 9523 log.cc:824] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:06.609563 9523 tablet_bootstrap.cc:492] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: No bootstrap required, opened a new log
I20260620 14:53:06.609742 9523 ts_tablet_manager.cc:1397] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 14:53:06.609576 9519 raft_consensus.cc:348] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } }
I20260620 14:53:06.609845 9519 raft_consensus.cc:374] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:06.609912 9519 raft_consensus.cc:729] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 274bb23d20a44efe95b8fdca492306b8, State: Initialized, Role: FOLLOWER
I20260620 14:53:06.610205 9520 raft_consensus.cc:348] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } }
I20260620 14:53:06.610440 9520 raft_consensus.cc:374] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:06.610391 9519 consensus_queue.cc:260] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } }
I20260620 14:53:06.610515 9520 raft_consensus.cc:729] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52fe6429041c498db60532f938ea38e7, State: Initialized, Role: FOLLOWER
I20260620 14:53:06.610924 9520 consensus_queue.cc:260] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } }
I20260620 14:53:06.611439 9519 ts_tablet_manager.cc:1428] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260620 14:53:06.611840 9520 ts_tablet_manager.cc:1428] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260620 14:53:06.613034 9523 raft_consensus.cc:348] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } }
I20260620 14:53:06.613240 9523 raft_consensus.cc:374] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:06.613304 9523 raft_consensus.cc:729] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c68ae7817e040729aaa92be7337ddc8, State: Initialized, Role: FOLLOWER
I20260620 14:53:06.613490 9356 heartbeater.cc:503] Master 127.7.24.254:33969 was elected leader, sending a full tablet report...
I20260620 14:53:06.613657 9523 consensus_queue.cc:260] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } }
I20260620 14:53:06.614291 9496 heartbeater.cc:503] Master 127.7.24.254:33969 was elected leader, sending a full tablet report...
I20260620 14:53:06.614544 9523 ts_tablet_manager.cc:1428] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 14:53:06.614605 9216 heartbeater.cc:503] Master 127.7.24.254:33969 was elected leader, sending a full tablet report...
I20260620 14:53:06.616135 9526 raft_consensus.cc:482] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:06.616281 9526 raft_consensus.cc:504] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } }
I20260620 14:53:06.617477 9526 leader_election.cc:288] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195:43839), 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381)
I20260620 14:53:06.621291 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.618183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:38889 (local address 127.7.24.195:43839)
0620 14:53:06.618294 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.618299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.618313 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:06.618353 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.618356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.618400 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.618508 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:06.618780 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.619296 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.620315 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.620726 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.620853 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.620907 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.621106 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.621170 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 14:53:06.621292 9249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.617645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:43839 (local address 127.7.24.194:38889)
0620 14:53:06.617897 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.617914 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.617995 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.618518 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.618521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.618538 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:06.618657 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.618663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.619407 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.619412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.620192 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.620202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.620840 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.620844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.620976 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.621003 (+ 27us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.621093 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.621163 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":21}
I20260620 14:53:06.621821 9450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0560c37074a4c1fa186c9d780bfe3bc" candidate_uuid: "52fe6429041c498db60532f938ea38e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c68ae7817e040729aaa92be7337ddc8" is_pre_election: true
I20260620 14:53:06.622047 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.617774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33381 (local address 127.7.24.194:35515)
0620 14:53:06.618600 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.618614 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.618713 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.618946 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.618948 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.618962 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:06.619395 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.619401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.620086 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.620092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.620872 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.620880 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.621633 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.621638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.621744 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.621767 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.621848 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.621914 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":2,"thread_start_us":117,"threads_started":1}
I20260620 14:53:06.622068 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.618183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:35515 (local address 127.7.24.193:33381)
0620 14:53:06.618294 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.618299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.618680 (+ 381us) server_negotiation.cc:407] Connection header received
0620 14:53:06.618803 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.618805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.618846 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.618939 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:06.619509 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.619982 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.620987 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.621505 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.621617 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.621659 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.621868 (+ 209us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.621945 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 14:53:06.622092 9450 raft_consensus.cc:2454] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52fe6429041c498db60532f938ea38e7 in term 0.
I20260620 14:53:06.622536 9243 leader_election.cc:302] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52fe6429041c498db60532f938ea38e7, 6c68ae7817e040729aaa92be7337ddc8; no voters:
I20260620 14:53:06.622677 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0560c37074a4c1fa186c9d780bfe3bc" candidate_uuid: "52fe6429041c498db60532f938ea38e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274bb23d20a44efe95b8fdca492306b8" is_pre_election: true
I20260620 14:53:06.622786 9526 raft_consensus.cc:2781] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:06.622901 9526 raft_consensus.cc:482] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:06.622956 9526 raft_consensus.cc:3037] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:06.622931 9170 raft_consensus.cc:2454] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52fe6429041c498db60532f938ea38e7 in term 0.
I20260620 14:53:06.624001 9526 raft_consensus.cc:504] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } }
I20260620 14:53:06.624336 9526 leader_election.cc:288] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [CANDIDATE]: Term 1 election: Requested vote from peers 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195:43839), 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381)
I20260620 14:53:06.624682 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0560c37074a4c1fa186c9d780bfe3bc" candidate_uuid: "52fe6429041c498db60532f938ea38e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274bb23d20a44efe95b8fdca492306b8"
I20260620 14:53:06.624682 9450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0560c37074a4c1fa186c9d780bfe3bc" candidate_uuid: "52fe6429041c498db60532f938ea38e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c68ae7817e040729aaa92be7337ddc8"
I20260620 14:53:06.624800 9170 raft_consensus.cc:3037] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:06.624828 9450 raft_consensus.cc:3037] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:06.626020 9170 raft_consensus.cc:2454] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52fe6429041c498db60532f938ea38e7 in term 1.
I20260620 14:53:06.626020 9450 raft_consensus.cc:2454] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52fe6429041c498db60532f938ea38e7 in term 1.
I20260620 14:53:06.626343 9241 leader_election.cc:302] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274bb23d20a44efe95b8fdca492306b8, 52fe6429041c498db60532f938ea38e7; no voters:
I20260620 14:53:06.626530 9526 raft_consensus.cc:2781] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:06.626751 9526 raft_consensus.cc:686] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 1 LEADER]: Becoming Leader. State: Replica: 52fe6429041c498db60532f938ea38e7, State: Running, Role: LEADER
I20260620 14:53:06.627044 9526 consensus_queue.cc:237] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } }
I20260620 14:53:06.630024 9012 catalog_manager.cc:5495] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52fe6429041c498db60532f938ea38e7 (127.7.24.194). New cstate: current_term: 1 leader_uuid: "52fe6429041c498db60532f938ea38e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } health_report { overall_health: UNKNOWN } } }
Jun 20 14:53:06 dist-test-slave-sms9 krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967185, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:06.668722 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.660328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.7.24.254:33969)
0620 14:53:06.660470 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.660474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.660697 (+ 223us) server_negotiation.cc:407] Connection header received
0620 14:53:06.660822 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.660824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.660864 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.660944 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:06.661584 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.662131 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.662948 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.663188 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.663293 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.665532 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:06.665535 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:06.665537 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:06.665558 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:06.667089 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.667521 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.667524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.667525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.667593 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:06.667889 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:06.667892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:06.667893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:06.668059 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:06.668132 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.668456 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.668591 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":56}
I20260620 14:53:06.668759 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.660271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:33969 (local address 127.0.0.1:55506)
0620 14:53:06.660591 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.660608 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.660729 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.660985 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.660987 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.661379 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:06.661473 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.661479 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.662228 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.662230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.662795 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.662803 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.663296 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.663299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.663371 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.664005 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:06.664017 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:06.665406 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:06.667193 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.667202 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.667209 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.667434 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.667711 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:06.667714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:06.667715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:06.667803 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:06.668177 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:06.668183 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:06.668296 (+ 113us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.668406 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.668541 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":148,"threads_started":1}
I20260620 14:53:06.673221 9012 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:55506:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 14:53:06.673996 9012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:06.679175 9150 tablet_service.cc:1467] Processing CreateTablet for tablet e2b570f8eb744765b4c2b03e9b7ea4cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c8b15310200e430894b6286ac76f3ce5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 14:53:06.679183 9290 tablet_service.cc:1467] Processing CreateTablet for tablet e2b570f8eb744765b4c2b03e9b7ea4cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c8b15310200e430894b6286ac76f3ce5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 14:53:06.679961 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2b570f8eb744765b4c2b03e9b7ea4cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:06.679960 9290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2b570f8eb744765b4c2b03e9b7ea4cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:06.679657 9418 tablet_service.cc:1467] Processing CreateTablet for tablet e2b570f8eb744765b4c2b03e9b7ea4cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c8b15310200e430894b6286ac76f3ce5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 14:53:06.680250 9418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2b570f8eb744765b4c2b03e9b7ea4cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:06.682547 9523 tablet_bootstrap.cc:492] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: Bootstrap starting.
I20260620 14:53:06.682834 9520 tablet_bootstrap.cc:492] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7: Bootstrap starting.
I20260620 14:53:06.683605 9523 tablet_bootstrap.cc:654] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:06.683807 9520 tablet_bootstrap.cc:654] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:06.684444 9519 tablet_bootstrap.cc:492] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8: Bootstrap starting.
I20260620 14:53:06.685139 9523 tablet_bootstrap.cc:492] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: No bootstrap required, opened a new log
I20260620 14:53:06.685240 9523 ts_tablet_manager.cc:1397] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:06.685483 9519 tablet_bootstrap.cc:654] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:06.685561 9520 tablet_bootstrap.cc:492] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7: No bootstrap required, opened a new log
I20260620 14:53:06.685652 9520 ts_tablet_manager.cc:1397] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260620 14:53:06.685750 9523 raft_consensus.cc:348] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.685875 9523 raft_consensus.cc:374] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:06.685918 9523 raft_consensus.cc:729] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c68ae7817e040729aaa92be7337ddc8, State: Initialized, Role: FOLLOWER
I20260620 14:53:06.686046 9523 consensus_queue.cc:260] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.686120 9520 raft_consensus.cc:348] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.686229 9520 raft_consensus.cc:374] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:06.686261 9520 raft_consensus.cc:729] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52fe6429041c498db60532f938ea38e7, State: Initialized, Role: FOLLOWER
I20260620 14:53:06.686375 9520 consensus_queue.cc:260] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.686338 9527 tablet_replica.cc:439] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } }
I20260620 14:53:06.686535 9527 tablet_replica.cc:442] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:06.686641 9526 tablet_replica.cc:439] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } }
I20260620 14:53:06.686760 9526 tablet_replica.cc:442] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:06.686646 9520 ts_tablet_manager.cc:1428] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:06.687538 9519 tablet_bootstrap.cc:492] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8: No bootstrap required, opened a new log
I20260620 14:53:06.687597 9519 ts_tablet_manager.cc:1397] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 14:53:06.687944 9519 raft_consensus.cc:348] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.688021 9519 raft_consensus.cc:374] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:06.688041 9519 raft_consensus.cc:729] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 274bb23d20a44efe95b8fdca492306b8, State: Initialized, Role: FOLLOWER
I20260620 14:53:06.688161 9519 consensus_queue.cc:260] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.688422 9519 ts_tablet_manager.cc:1428] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 14:53:06.688439 9525 tablet_replica.cc:439] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } }
I20260620 14:53:06.688599 9525 tablet_replica.cc:442] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:06.686337 9523 ts_tablet_manager.cc:1428] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 14:53:06.711203 9527 raft_consensus.cc:482] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:06.711354 9527 raft_consensus.cc:504] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.712570 9527 leader_election.cc:288] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381), 52fe6429041c498db60532f938ea38e7 (127.7.24.194:38547)
I20260620 14:53:06.716344 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.712800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38547 (local address 127.7.24.195:50199)
0620 14:53:06.712917 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.712932 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.713018 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.713282 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.713286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.713306 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:06.713436 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.713442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.714371 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.714376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.715227 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.715236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.715889 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.715892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.716014 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.716045 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.716140 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.716211 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":25}
I20260620 14:53:06.716344 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.712917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:50199 (local address 127.7.24.194:38547)
0620 14:53:06.713055 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.713058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.713072 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:06.713118 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.713120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.713159 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.713232 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:06.713558 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.714260 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.715356 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.715760 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.715896 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.715947 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.716155 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.716231 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 14:53:06.716593 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.712925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:44913 (local address 127.7.24.193:33381)
0620 14:53:06.713039 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.713044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.713543 (+ 499us) server_negotiation.cc:407] Connection header received
0620 14:53:06.713684 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.713687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.713728 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.713793 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:06.714068 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.714603 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.715645 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.716056 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.716187 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.716243 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.716410 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.716475 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 14:53:06.716614 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.712805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33381 (local address 127.7.24.195:44913)
0620 14:53:06.713452 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.713464 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.713587 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.713834 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.713837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.713854 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:06.713968 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.713974 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.714705 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.714710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.715524 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.715532 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.716187 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.716190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.716284 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.716311 (+ 27us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.716396 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.716475 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":311,"thread_start_us":162,"threads_started":1}
I20260620 14:53:06.717011 9310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b570f8eb744765b4c2b03e9b7ea4cf" candidate_uuid: "6c68ae7817e040729aaa92be7337ddc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52fe6429041c498db60532f938ea38e7" is_pre_election: true
I20260620 14:53:06.717192 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b570f8eb744765b4c2b03e9b7ea4cf" candidate_uuid: "6c68ae7817e040729aaa92be7337ddc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274bb23d20a44efe95b8fdca492306b8" is_pre_election: true
I20260620 14:53:06.717268 9310 raft_consensus.cc:2454] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c68ae7817e040729aaa92be7337ddc8 in term 0.
I20260620 14:53:06.717337 9170 raft_consensus.cc:2454] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c68ae7817e040729aaa92be7337ddc8 in term 0.
I20260620 14:53:06.717705 9381 leader_election.cc:302] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274bb23d20a44efe95b8fdca492306b8, 6c68ae7817e040729aaa92be7337ddc8; no voters:
I20260620 14:53:06.718051 9527 raft_consensus.cc:2781] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:06.718142 9527 raft_consensus.cc:482] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:06.718199 9527 raft_consensus.cc:3037] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:06.719234 9527 raft_consensus.cc:504] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.719626 9527 leader_election.cc:288] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [CANDIDATE]: Term 1 election: Requested vote from peers 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381), 52fe6429041c498db60532f938ea38e7 (127.7.24.194:38547)
I20260620 14:53:06.719923 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b570f8eb744765b4c2b03e9b7ea4cf" candidate_uuid: "6c68ae7817e040729aaa92be7337ddc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274bb23d20a44efe95b8fdca492306b8"
I20260620 14:53:06.719942 9310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2b570f8eb744765b4c2b03e9b7ea4cf" candidate_uuid: "6c68ae7817e040729aaa92be7337ddc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52fe6429041c498db60532f938ea38e7"
I20260620 14:53:06.720049 9310 raft_consensus.cc:3037] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:06.720050 9170 raft_consensus.cc:3037] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:06.721221 9170 raft_consensus.cc:2454] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c68ae7817e040729aaa92be7337ddc8 in term 1.
I20260620 14:53:06.721235 9310 raft_consensus.cc:2454] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c68ae7817e040729aaa92be7337ddc8 in term 1.
I20260620 14:53:06.721530 9382 leader_election.cc:302] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52fe6429041c498db60532f938ea38e7, 6c68ae7817e040729aaa92be7337ddc8; no voters:
I20260620 14:53:06.721696 9527 raft_consensus.cc:2781] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:06.721946 9527 raft_consensus.cc:686] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [term 1 LEADER]: Becoming Leader. State: Replica: 6c68ae7817e040729aaa92be7337ddc8, State: Running, Role: LEADER
I20260620 14:53:06.722311 9527 consensus_queue.cc:237] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.724563 9541 tablet_replica.cc:439] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: TxnStatusTablet state changed. Reason: New leader 6c68ae7817e040729aaa92be7337ddc8. Latest consensus state: current_term: 1 leader_uuid: "6c68ae7817e040729aaa92be7337ddc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } }
I20260620 14:53:06.724680 9541 tablet_replica.cc:442] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: This TxnStatusTablet replica's current role is: LEADER
I20260620 14:53:06.725008 9543 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 14:53:06.725258 9543 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 14:53:06.725991 9013 catalog_manager.cc:5495] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195). New cstate: current_term: 1 leader_uuid: "6c68ae7817e040729aaa92be7337ddc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } health_report { overall_health: HEALTHY } } }
I20260620 14:53:06.746093 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.742331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33112 (local address 127.7.24.195:43839)
0620 14:53:06.742491 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.742496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.742510 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:06.742680 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.742683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.742743 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.742844 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:06.743079 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.743639 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.744546 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.744829 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.744946 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.745266 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:06.745659 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:06.745786 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.745871 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.745929 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 14:53:06.746068 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.742199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:43839 (local address 127.0.0.1:33112)
0620 14:53:06.742342 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.742392 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.742532 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.742856 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.742859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.742867 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 14:53:06.742977 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.742983 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.743741 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.743746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.744428 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.744437 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.744943 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.744946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.745068 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.745128 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 14:53:06.745778 (+ 650us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 14:53:06.745795 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.745861 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.745942 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 14:53:06.751909 9310 raft_consensus.cc:1261] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Refusing update from remote peer 6c68ae7817e040729aaa92be7337ddc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:06.751937 9170 raft_consensus.cc:1261] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Refusing update from remote peer 6c68ae7817e040729aaa92be7337ddc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:06.752630 9527 consensus_queue.cc:1035] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:06.752791 9541 consensus_queue.cc:1035] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:06.756148 9526 tablet_replica.cc:439] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7: TxnStatusTablet state changed. Reason: New leader 6c68ae7817e040729aaa92be7337ddc8. Latest consensus state: current_term: 1 leader_uuid: "6c68ae7817e040729aaa92be7337ddc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } }
I20260620 14:53:06.756273 9526 tablet_replica.cc:442] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:06.757771 9525 tablet_replica.cc:439] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8: TxnStatusTablet state changed. Reason: New leader 6c68ae7817e040729aaa92be7337ddc8. Latest consensus state: current_term: 1 leader_uuid: "6c68ae7817e040729aaa92be7337ddc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } }
I20260620 14:53:06.757897 9525 tablet_replica.cc:442] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:06.760182 9527 tablet_replica.cc:439] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c68ae7817e040729aaa92be7337ddc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } }
I20260620 14:53:06.760313 9527 tablet_replica.cc:442] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: This TxnStatusTablet replica's current role is: LEADER
I20260620 14:53:06.760588 9526 tablet_replica.cc:439] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c68ae7817e040729aaa92be7337ddc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } }
I20260620 14:53:06.760702 9526 tablet_replica.cc:442] T e2b570f8eb744765b4c2b03e9b7ea4cf P 52fe6429041c498db60532f938ea38e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:06.762735 9546 mvcc.cc:205] Tried to move back new op lower bound from 7298937596928585728 to 7298937596817371136. Current Snapshot: MvccSnapshot[applied={T|T < 7298937596928585728}]
I20260620 14:53:06.764663 9541 tablet_replica.cc:439] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c68ae7817e040729aaa92be7337ddc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } }
I20260620 14:53:06.764782 9541 tablet_replica.cc:442] T e2b570f8eb744765b4c2b03e9b7ea4cf P 6c68ae7817e040729aaa92be7337ddc8: This TxnStatusTablet replica's current role is: LEADER
I20260620 14:53:06.765990 9525 tablet_replica.cc:439] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c68ae7817e040729aaa92be7337ddc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } } }
I20260620 14:53:06.766098 9525 tablet_replica.cc:442] T e2b570f8eb744765b4c2b03e9b7ea4cf P 274bb23d20a44efe95b8fdca492306b8: This TxnStatusTablet replica's current role is: FOLLOWER
W20260620 14:53:06.767215 9357 tablet.cc:2369] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:53:06.771051 9547 mvcc.cc:205] Tried to move back new op lower bound from 7298937596928585728 to 7298937596817371136. Current Snapshot: MvccSnapshot[applied={T|T < 7298937596928585728}]
W20260620 14:53:06.773137 9497 tablet.cc:2369] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:53:06.775586 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.771356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58546 (local address 127.7.24.194:38547)
0620 14:53:06.771491 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.771495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.771508 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:53:06.771573 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.771576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.771635 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.771730 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:06.772123 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.773086 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.774102 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.774343 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.774460 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.774813 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:06.775192 (+ 379us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:06.775320 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.775397 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.775449 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 14:53:06.786733 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.783557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:43839 (local address 127.7.24.194:59099)
0620 14:53:06.783714 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.783731 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.783819 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.784058 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.784061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.784068 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 14:53:06.784179 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.784184 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.784897 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.784903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.785504 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.785512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.785954 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.785957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.786051 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.786090 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 14:53:06.786394 (+ 304us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 14:53:06.786402 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.786513 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.786620 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 14:53:06.786706 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.783665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:59099 (local address 127.7.24.195:43839)
0620 14:53:06.783813 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.783818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.783832 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:06.783908 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.783911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.783962 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.784038 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:06.784297 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.784791 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.785624 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.785846 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.785955 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.786245 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:06.786310 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:06.786390 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.786523 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.786572 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260620 14:53:06.792142 9217 tablet.cc:2369] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:53:06.795202 9450 raft_consensus.cc:1261] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Refusing update from remote peer 52fe6429041c498db60532f938ea38e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:06.795280 9170 raft_consensus.cc:1261] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Refusing update from remote peer 52fe6429041c498db60532f938ea38e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:06.795760 9529 consensus_queue.cc:1035] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:06.795961 9526 consensus_queue.cc:1035] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:06.820465 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.816558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38547 (local address 127.7.24.195:59263)
0620 14:53:06.816698 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.816714 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.816797 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.817117 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.817121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.817130 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 14:53:06.817259 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.817266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.818191 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.818194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.819102 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.819111 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.819785 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.819789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.819880 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.819922 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 14:53:06.820245 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 14:53:06.820251 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.820300 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.820361 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260620 14:53:06.820515 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.816625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:59263 (local address 127.7.24.194:38547)
0620 14:53:06.816924 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.816927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.816936 (+ 9us) server_negotiation.cc:407] Connection header received
0620 14:53:06.816968 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.816969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.817006 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.817060 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:06.817397 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.818066 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.819218 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.819605 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.819718 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.820054 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:06.820138 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:06.820227 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.820325 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.820379 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236}
I20260620 14:53:06.844734 9013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:06.844945 9013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:06.847746 9013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33: Sending DeleteTablet for 3 replicas of tablet d0560c37074a4c1fa186c9d780bfe3bc
I20260620 14:53:06.848558 9418 tablet_service.cc:1514] Processing DeleteTablet for tablet d0560c37074a4c1fa186c9d780bfe3bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:06 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:33102
I20260620 14:53:06.848721 9150 tablet_service.cc:1514] Processing DeleteTablet for tablet d0560c37074a4c1fa186c9d780bfe3bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:06 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:58590
I20260620 14:53:06.849210 9565 tablet_replica.cc:330] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: stopping tablet replica
I20260620 14:53:06.849586 9565 raft_consensus.cc:2229] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:06.849677 9290 tablet_service.cc:1514] Processing DeleteTablet for tablet d0560c37074a4c1fa186c9d780bfe3bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:06 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:58532
I20260620 14:53:06.849897 9565 raft_consensus.cc:2258] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:06.850817 9013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:06.851257 9013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:06.852011 9565 ts_tablet_manager.cc:1905] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:06.855008 9567 tablet_replica.cc:330] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: stopping tablet replica
I20260620 14:53:06.855533 9565 ts_tablet_manager.cc:1918] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:06.855719 9565 log.cc:1196] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/d0560c37074a4c1fa186c9d780bfe3bc
I20260620 14:53:06.856096 9565 ts_tablet_manager.cc:1939] T d0560c37074a4c1fa186c9d780bfe3bc P 6c68ae7817e040729aaa92be7337ddc8: Deleting consensus metadata
I20260620 14:53:06.857692 9418 tablet_service.cc:1467] Processing CreateTablet for tablet 29fc9616d80540b98c475c9a7b47edc4 (DEFAULT_TABLE table=test-table [id=ef035a7713444d79ba1d9fcf57dd2247]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:06.857726 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 29fc9616d80540b98c475c9a7b47edc4 (DEFAULT_TABLE table=test-table [id=ef035a7713444d79ba1d9fcf57dd2247]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:06.857947 9418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29fc9616d80540b98c475c9a7b47edc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:06.858058 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29fc9616d80540b98c475c9a7b47edc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:06.860383 9523 tablet_bootstrap.cc:492] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8: Bootstrap starting.
I20260620 14:53:06.860952 9290 tablet_service.cc:1467] Processing CreateTablet for tablet 29fc9616d80540b98c475c9a7b47edc4 (DEFAULT_TABLE table=test-table [id=ef035a7713444d79ba1d9fcf57dd2247]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:06.861253 9290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29fc9616d80540b98c475c9a7b47edc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:06.861306 9523 tablet_bootstrap.cc:654] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:06.861904 9519 tablet_bootstrap.cc:492] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8: Bootstrap starting.
I20260620 14:53:06.862820 9519 tablet_bootstrap.cc:654] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:06.863955 8999 catalog_manager.cc:4841] TS 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195:43839): tablet d0560c37074a4c1fa186c9d780bfe3bc (table test-table [id=b521637adcb94e208d433a79867bf356]) successfully deleted
I20260620 14:53:06.864243 9523 tablet_bootstrap.cc:492] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8: No bootstrap required, opened a new log
I20260620 14:53:06.864326 9523 ts_tablet_manager.cc:1397] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 14:53:06.864734 9520 tablet_bootstrap.cc:492] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7: Bootstrap starting.
I20260620 14:53:06.864748 9523 raft_consensus.cc:348] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.864850 9523 raft_consensus.cc:374] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:06.864881 9523 raft_consensus.cc:729] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c68ae7817e040729aaa92be7337ddc8, State: Initialized, Role: FOLLOWER
I20260620 14:53:06.865010 9523 consensus_queue.cc:260] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.865255 9523 ts_tablet_manager.cc:1428] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:06.865752 9520 tablet_bootstrap.cc:654] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:06.867079 9520 tablet_bootstrap.cc:492] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7: No bootstrap required, opened a new log
I20260620 14:53:06.867159 9520 ts_tablet_manager.cc:1397] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:06.867579 9520 raft_consensus.cc:348] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.867681 9520 raft_consensus.cc:374] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:06.867709 9520 raft_consensus.cc:729] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52fe6429041c498db60532f938ea38e7, State: Initialized, Role: FOLLOWER
I20260620 14:53:06.867827 9520 consensus_queue.cc:260] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.868041 9520 ts_tablet_manager.cc:1428] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:06.871343 9519 tablet_bootstrap.cc:492] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8: No bootstrap required, opened a new log
I20260620 14:53:06.871366 9567 raft_consensus.cc:2229] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:06.871436 9519 ts_tablet_manager.cc:1397] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260620 14:53:06.871767 9567 raft_consensus.cc:2258] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:06.871894 9519 raft_consensus.cc:348] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.871999 9519 raft_consensus.cc:374] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:06.872030 9519 raft_consensus.cc:729] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 274bb23d20a44efe95b8fdca492306b8, State: Initialized, Role: FOLLOWER
I20260620 14:53:06.872160 9566 tablet_replica.cc:330] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: stopping tablet replica
I20260620 14:53:06.872155 9519 consensus_queue.cc:260] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.872473 9519 ts_tablet_manager.cc:1428] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:06.874116 9567 ts_tablet_manager.cc:1905] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:06.875490 9566 raft_consensus.cc:2229] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:06.875744 9566 raft_consensus.cc:2258] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:06.877311 9566 ts_tablet_manager.cc:1905] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:06.877775 9567 ts_tablet_manager.cc:1918] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:06.877913 9567 log.cc:1196] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/d0560c37074a4c1fa186c9d780bfe3bc
I20260620 14:53:06.878363 9567 ts_tablet_manager.cc:1939] T d0560c37074a4c1fa186c9d780bfe3bc P 52fe6429041c498db60532f938ea38e7: Deleting consensus metadata
I20260620 14:53:06.879755 8998 catalog_manager.cc:4841] TS 52fe6429041c498db60532f938ea38e7 (127.7.24.194:38547): tablet d0560c37074a4c1fa186c9d780bfe3bc (table test-table [id=b521637adcb94e208d433a79867bf356]) successfully deleted
I20260620 14:53:06.881453 9566 ts_tablet_manager.cc:1918] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:06.881562 9566 log.cc:1196] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/wal/wals/d0560c37074a4c1fa186c9d780bfe3bc
I20260620 14:53:06.882215 9566 ts_tablet_manager.cc:1939] T d0560c37074a4c1fa186c9d780bfe3bc P 274bb23d20a44efe95b8fdca492306b8: Deleting consensus metadata
I20260620 14:53:06.883600 8997 catalog_manager.cc:4841] TS 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381): tablet d0560c37074a4c1fa186c9d780bfe3bc (table test-table [id=b521637adcb94e208d433a79867bf356]) successfully deleted
I20260620 14:53:06.929913 9560 raft_consensus.cc:482] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:06.930110 9560 raft_consensus.cc:504] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.930697 9560 leader_election.cc:288] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381), 52fe6429041c498db60532f938ea38e7 (127.7.24.194:38547)
I20260620 14:53:06.931139 9310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fc9616d80540b98c475c9a7b47edc4" candidate_uuid: "6c68ae7817e040729aaa92be7337ddc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52fe6429041c498db60532f938ea38e7" is_pre_election: true
I20260620 14:53:06.931128 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fc9616d80540b98c475c9a7b47edc4" candidate_uuid: "6c68ae7817e040729aaa92be7337ddc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274bb23d20a44efe95b8fdca492306b8" is_pre_election: true
I20260620 14:53:06.931282 9170 raft_consensus.cc:2454] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c68ae7817e040729aaa92be7337ddc8 in term 0.
I20260620 14:53:06.931282 9310 raft_consensus.cc:2454] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c68ae7817e040729aaa92be7337ddc8 in term 0.
I20260620 14:53:06.931627 9381 leader_election.cc:302] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274bb23d20a44efe95b8fdca492306b8, 6c68ae7817e040729aaa92be7337ddc8; no voters:
I20260620 14:53:06.931783 9560 raft_consensus.cc:2781] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:06.931838 9560 raft_consensus.cc:482] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:06.931881 9560 raft_consensus.cc:3037] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:06.932989 9560 raft_consensus.cc:504] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.933393 9560 leader_election.cc:288] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [CANDIDATE]: Term 1 election: Requested vote from peers 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381), 52fe6429041c498db60532f938ea38e7 (127.7.24.194:38547)
I20260620 14:53:06.933676 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fc9616d80540b98c475c9a7b47edc4" candidate_uuid: "6c68ae7817e040729aaa92be7337ddc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274bb23d20a44efe95b8fdca492306b8"
I20260620 14:53:06.933679 9310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fc9616d80540b98c475c9a7b47edc4" candidate_uuid: "6c68ae7817e040729aaa92be7337ddc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52fe6429041c498db60532f938ea38e7"
I20260620 14:53:06.933800 9310 raft_consensus.cc:3037] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:06.933800 9170 raft_consensus.cc:3037] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:06.934963 9170 raft_consensus.cc:2454] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c68ae7817e040729aaa92be7337ddc8 in term 1.
I20260620 14:53:06.934963 9310 raft_consensus.cc:2454] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c68ae7817e040729aaa92be7337ddc8 in term 1.
I20260620 14:53:06.935243 9382 leader_election.cc:302] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52fe6429041c498db60532f938ea38e7, 6c68ae7817e040729aaa92be7337ddc8; no voters:
I20260620 14:53:06.935459 9560 raft_consensus.cc:2781] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:06.935557 9560 raft_consensus.cc:686] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 1 LEADER]: Becoming Leader. State: Replica: 6c68ae7817e040729aaa92be7337ddc8, State: Running, Role: LEADER
I20260620 14:53:06.935716 9560 consensus_queue.cc:237] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:06.937409 9013 catalog_manager.cc:5495] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195). New cstate: current_term: 1 leader_uuid: "6c68ae7817e040729aaa92be7337ddc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } health_report { overall_health: HEALTHY } } }
I20260620 14:53:06.961136 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.957458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.7.24.195:43839)
0620 14:53:06.957590 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.957595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.957611 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:53:06.957717 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.957722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.957775 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.957859 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:06.958282 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.959099 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.959942 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.960309 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.960422 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.960636 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:06.960721 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:06.960793 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.960937 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.960991 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 14:53:06.970984 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.967580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:42955 (local address 127.7.24.195:43839)
0620 14:53:06.967705 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:06.967709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:06.967723 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:06.967763 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:06.967766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:06.967813 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:06.967897 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:06.968206 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.968953 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.969832 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.970138 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.970249 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.970502 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:06.970582 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:06.970662 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:06.970796 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 14:53:06.970848 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:53:06.971007 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:06.967391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:43839 (local address 127.7.24.195:42955)
0620 14:53:06.967538 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:06.967555 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:06.967640 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:06.967942 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:06.967944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:06.967950 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 14:53:06.968037 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.968041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.969077 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.969080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.969654 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:06.969662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:06.970259 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:06.970262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:06.970320 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:06.970346 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 14:53:06.970688 (+ 342us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 14:53:06.970695 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 14:53:06.970757 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 14:53:06.970832 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260620 14:53:06.977226 9170 raft_consensus.cc:1261] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Refusing update from remote peer 6c68ae7817e040729aaa92be7337ddc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:06.977295 9310 raft_consensus.cc:1261] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Refusing update from remote peer 6c68ae7817e040729aaa92be7337ddc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:06.977679 9527 consensus_queue.cc:1035] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:06.977803 9541 consensus_queue.cc:1035] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:06.981026 9548 mvcc.cc:205] Tried to move back new op lower bound from 7298937597855395840 to 7298937597689962496. Current Snapshot: MvccSnapshot[applied={T|T < 7298937597855395840}]
W20260620 14:53:06.995165 9546 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 14:53:08.556602 9013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:08.556764 9013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:08.559289 9013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33: Sending DeleteTablet for 3 replicas of tablet 29fc9616d80540b98c475c9a7b47edc4
I20260620 14:53:08.559983 9418 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fc9616d80540b98c475c9a7b47edc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:08 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:33102
I20260620 14:53:08.559990 9150 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fc9616d80540b98c475c9a7b47edc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:08 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:58590
I20260620 14:53:08.560053 9290 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fc9616d80540b98c475c9a7b47edc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:08 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:58532
I20260620 14:53:08.560554 9595 tablet_replica.cc:330] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7: stopping tablet replica
I20260620 14:53:08.560724 9595 raft_consensus.cc:2229] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:08.560855 9595 raft_consensus.cc:2258] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:08.561105 9594 tablet_replica.cc:330] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8: stopping tablet replica
I20260620 14:53:08.561232 9594 raft_consensus.cc:2229] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:08.561545 9594 raft_consensus.cc:2258] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:08.561578 9595 ts_tablet_manager.cc:1905] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:08.561681 9596 tablet_replica.cc:330] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8: stopping tablet replica
I20260620 14:53:08.561813 9596 raft_consensus.cc:2229] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:08.561952 9596 raft_consensus.cc:2258] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:08.562613 9596 ts_tablet_manager.cc:1905] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:08.562613 9594 ts_tablet_manager.cc:1905] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:08.565039 9595 ts_tablet_manager.cc:1918] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 14:53:08.565090 9594 ts_tablet_manager.cc:1918] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 14:53:08.565090 9596 ts_tablet_manager.cc:1918] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 14:53:08.565160 9596 log.cc:1196] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/wal/wals/29fc9616d80540b98c475c9a7b47edc4
I20260620 14:53:08.565160 9594 log.cc:1196] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/29fc9616d80540b98c475c9a7b47edc4
I20260620 14:53:08.565171 9595 log.cc:1196] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/29fc9616d80540b98c475c9a7b47edc4
I20260620 14:53:08.565448 9595 ts_tablet_manager.cc:1939] T 29fc9616d80540b98c475c9a7b47edc4 P 52fe6429041c498db60532f938ea38e7: Deleting consensus metadata
I20260620 14:53:08.565477 9594 ts_tablet_manager.cc:1939] T 29fc9616d80540b98c475c9a7b47edc4 P 6c68ae7817e040729aaa92be7337ddc8: Deleting consensus metadata
I20260620 14:53:08.565477 9596 ts_tablet_manager.cc:1939] T 29fc9616d80540b98c475c9a7b47edc4 P 274bb23d20a44efe95b8fdca492306b8: Deleting consensus metadata
I20260620 14:53:08.566298 8999 catalog_manager.cc:4841] TS 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195:43839): tablet 29fc9616d80540b98c475c9a7b47edc4 (table test-table [id=ef035a7713444d79ba1d9fcf57dd2247]) successfully deleted
I20260620 14:53:08.566298 8998 catalog_manager.cc:4841] TS 52fe6429041c498db60532f938ea38e7 (127.7.24.194:38547): tablet 29fc9616d80540b98c475c9a7b47edc4 (table test-table [id=ef035a7713444d79ba1d9fcf57dd2247]) successfully deleted
I20260620 14:53:08.566483 8997 catalog_manager.cc:4841] TS 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381): tablet 29fc9616d80540b98c475c9a7b47edc4 (table test-table [id=ef035a7713444d79ba1d9fcf57dd2247]) successfully deleted
Jun 20 14:53:08 dist-test-slave-sms9 krb5kdc[8968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967188, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 14:53:08 dist-test-slave-sms9 krb5kdc[8968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967188, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:08.596608 9604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:08.582323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55510 (local address 127.7.24.254:33969)
0620 14:53:08.582606 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:08.582612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:08.582627 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:53:08.582770 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:08.582773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:08.582816 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:08.582903 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:08.583603 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.584147 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.584943 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.585168 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.585248 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:08.593619 (+ 8371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:08.593623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:08.593625 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:08.593653 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:08.595123 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:08.595544 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:08.595546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:08.595547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:08.595621 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:08.595857 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:08.595859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:08.595860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:08.596038 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:08.596137 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:08.596367 (+ 230us) server_negotiation.cc:299] Negotiation successful
0620 14:53:08.596473 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":104,"threads_started":1}
I20260620 14:53:08.599259 9013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:08.599643 9013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:08.604249 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 5a6c20e004464c849d55164a213d887f (DEFAULT_TABLE table=test-table [id=1277553096444f8bac08870972599113]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:08.604290 9290 tablet_service.cc:1467] Processing CreateTablet for tablet 5a6c20e004464c849d55164a213d887f (DEFAULT_TABLE table=test-table [id=1277553096444f8bac08870972599113]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:08.604553 9290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a6c20e004464c849d55164a213d887f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:08.604554 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a6c20e004464c849d55164a213d887f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:08.604487 9418 tablet_service.cc:1467] Processing CreateTablet for tablet 5a6c20e004464c849d55164a213d887f (DEFAULT_TABLE table=test-table [id=1277553096444f8bac08870972599113]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:08.604734 9418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a6c20e004464c849d55164a213d887f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:08.607035 9606 tablet_bootstrap.cc:492] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8: Bootstrap starting.
I20260620 14:53:08.607112 9605 tablet_bootstrap.cc:492] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8: Bootstrap starting.
I20260620 14:53:08.607510 9607 tablet_bootstrap.cc:492] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7: Bootstrap starting.
I20260620 14:53:08.607957 9605 tablet_bootstrap.cc:654] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:08.607987 9606 tablet_bootstrap.cc:654] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:08.608438 9607 tablet_bootstrap.cc:654] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:08.609112 9605 tablet_bootstrap.cc:492] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8: No bootstrap required, opened a new log
I20260620 14:53:08.609257 9605 ts_tablet_manager.cc:1397] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:08.609752 9605 raft_consensus.cc:348] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.609851 9606 tablet_bootstrap.cc:492] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8: No bootstrap required, opened a new log
I20260620 14:53:08.609895 9605 raft_consensus.cc:374] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:08.609917 9607 tablet_bootstrap.cc:492] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7: No bootstrap required, opened a new log
I20260620 14:53:08.609941 9605 raft_consensus.cc:729] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c68ae7817e040729aaa92be7337ddc8, State: Initialized, Role: FOLLOWER
I20260620 14:53:08.609961 9606 ts_tablet_manager.cc:1397] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:08.609987 9607 ts_tablet_manager.cc:1397] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:08.610093 9605 consensus_queue.cc:260] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.610374 9606 raft_consensus.cc:348] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.610363 9607 raft_consensus.cc:348] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.610496 9606 raft_consensus.cc:374] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:08.610496 9605 ts_tablet_manager.cc:1428] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:08.610508 9607 raft_consensus.cc:374] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:08.610536 9606 raft_consensus.cc:729] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 274bb23d20a44efe95b8fdca492306b8, State: Initialized, Role: FOLLOWER
I20260620 14:53:08.610548 9607 raft_consensus.cc:729] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52fe6429041c498db60532f938ea38e7, State: Initialized, Role: FOLLOWER
I20260620 14:53:08.610697 9607 consensus_queue.cc:260] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.610688 9606 consensus_queue.cc:260] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.611057 9607 ts_tablet_manager.cc:1428] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:08.611064 9606 ts_tablet_manager.cc:1428] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260620 14:53:08.655056 9612 raft_consensus.cc:482] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:08.655225 9612 raft_consensus.cc:504] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.656493 9612 leader_election.cc:288] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52fe6429041c498db60532f938ea38e7 (127.7.24.194:38547), 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195:43839)
I20260620 14:53:08.662007 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:08.656843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:36539 (local address 127.7.24.195:43839)
0620 14:53:08.657121 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:08.657136 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:08.657152 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:53:08.657318 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:08.657321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:08.657374 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:08.657470 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:08.657826 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.658674 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.660551 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.661193 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.661323 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:08.661380 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:08.661771 (+ 391us) server_negotiation.cc:299] Negotiation successful
0620 14:53:08.661866 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":86,"threads_started":1}
I20260620 14:53:08.662007 9614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:08.656754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:59651 (local address 127.7.24.194:38547)
0620 14:53:08.657074 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:08.657084 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:08.657134 (+ 50us) server_negotiation.cc:407] Connection header received
0620 14:53:08.657318 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:08.657321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:08.657378 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:08.657470 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:08.657826 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.658680 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.660551 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.661193 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.661323 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:08.661383 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:08.661771 (+ 388us) server_negotiation.cc:299] Negotiation successful
0620 14:53:08.661856 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":113,"threads_started":1}
I20260620 14:53:08.662113 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:08.656654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38547 (local address 127.7.24.193:59651)
0620 14:53:08.657051 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:08.657067 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:08.657201 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:08.657503 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:08.657506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:08.657532 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:08.657709 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:08.657716 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.658820 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.658826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:08.660418 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:08.660430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.661342 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.661346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:08.661597 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:08.661653 (+ 56us) client_negotiation.cc:769] Sending connection context
0620 14:53:08.661739 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 14:53:08.661883 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":250,"spinlock_wait_cycles":12928,"thread_start_us":128,"threads_started":1}
I20260620 14:53:08.662554 9615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:08.656730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:43839 (local address 127.7.24.193:36539)
0620 14:53:08.657075 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:08.657087 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:08.657201 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:08.657503 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:08.657506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:08.657532 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:08.657710 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:08.657716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.658834 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.658837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:08.660417 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:08.660430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.661342 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.661346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:08.661624 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:08.661660 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 14:53:08.661739 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 14:53:08.661883 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":376,"spinlock_wait_cycles":30720,"thread_start_us":125,"threads_started":1}
I20260620 14:53:08.662576 9309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a6c20e004464c849d55164a213d887f" candidate_uuid: "274bb23d20a44efe95b8fdca492306b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52fe6429041c498db60532f938ea38e7" is_pre_election: true
I20260620 14:53:08.662735 9309 raft_consensus.cc:2454] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 274bb23d20a44efe95b8fdca492306b8 in term 0.
I20260620 14:53:08.663034 9450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a6c20e004464c849d55164a213d887f" candidate_uuid: "274bb23d20a44efe95b8fdca492306b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c68ae7817e040729aaa92be7337ddc8" is_pre_election: true
I20260620 14:53:08.663159 9450 raft_consensus.cc:2454] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 274bb23d20a44efe95b8fdca492306b8 in term 0.
I20260620 14:53:08.663594 9103 leader_election.cc:302] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 274bb23d20a44efe95b8fdca492306b8, 52fe6429041c498db60532f938ea38e7, 6c68ae7817e040729aaa92be7337ddc8; no voters:
I20260620 14:53:08.663938 9612 raft_consensus.cc:2781] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:08.663996 9612 raft_consensus.cc:482] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:08.664023 9612 raft_consensus.cc:3037] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:08.665035 9612 raft_consensus.cc:504] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.665400 9612 leader_election.cc:288] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [CANDIDATE]: Term 1 election: Requested vote from peers 52fe6429041c498db60532f938ea38e7 (127.7.24.194:38547), 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195:43839)
I20260620 14:53:08.665786 9309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a6c20e004464c849d55164a213d887f" candidate_uuid: "274bb23d20a44efe95b8fdca492306b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52fe6429041c498db60532f938ea38e7"
I20260620 14:53:08.665798 9450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a6c20e004464c849d55164a213d887f" candidate_uuid: "274bb23d20a44efe95b8fdca492306b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c68ae7817e040729aaa92be7337ddc8"
I20260620 14:53:08.665920 9450 raft_consensus.cc:3037] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:08.665918 9309 raft_consensus.cc:3037] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:08.667073 9450 raft_consensus.cc:2454] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 274bb23d20a44efe95b8fdca492306b8 in term 1.
I20260620 14:53:08.667073 9309 raft_consensus.cc:2454] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 274bb23d20a44efe95b8fdca492306b8 in term 1.
I20260620 14:53:08.667383 9103 leader_election.cc:302] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274bb23d20a44efe95b8fdca492306b8, 6c68ae7817e040729aaa92be7337ddc8; no voters:
I20260620 14:53:08.667560 9612 raft_consensus.cc:2781] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:08.667793 9612 raft_consensus.cc:686] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 1 LEADER]: Becoming Leader. State: Replica: 274bb23d20a44efe95b8fdca492306b8, State: Running, Role: LEADER
I20260620 14:53:08.668061 9612 consensus_queue.cc:237] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.670363 9013 catalog_manager.cc:5495] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193). New cstate: current_term: 1 leader_uuid: "274bb23d20a44efe95b8fdca492306b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } health_report { overall_health: UNKNOWN } } }
I20260620 14:53:08.696833 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:08.693417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58598 (local address 127.7.24.193:33381)
0620 14:53:08.693684 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:08.693695 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:08.693707 (+ 12us) server_negotiation.cc:407] Connection header received
0620 14:53:08.693777 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:08.693779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:08.693829 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:08.693938 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:08.694178 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.694769 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.695571 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.695795 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.695890 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:08.696091 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:08.696527 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:08.696613 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:08.696666 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 14:53:08.696710 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":118,"threads_started":1}
I20260620 14:53:08.706665 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:08.703561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:55523 (local address 127.7.24.195:43839)
0620 14:53:08.703685 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:08.703692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:08.703708 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:53:08.703872 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:08.703876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:08.703931 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:08.704016 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:08.704262 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.704755 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.705602 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.705833 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.705942 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:08.706223 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:08.706287 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:08.706367 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:08.706476 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 14:53:08.706526 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 14:53:08.706673 9623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:08.703351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:43839 (local address 127.7.24.193:55523)
0620 14:53:08.703641 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:08.703656 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:08.703749 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:08.704024 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:08.704027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:08.704034 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 14:53:08.704155 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:08.704159 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.704863 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.704876 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:08.705481 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:08.705489 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.705943 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.705947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:08.706046 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:08.706084 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 14:53:08.706372 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 14:53:08.706379 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 14:53:08.706445 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 14:53:08.706538 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":116,"threads_started":1}
I20260620 14:53:08.712879 9309 raft_consensus.cc:1261] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Refusing update from remote peer 274bb23d20a44efe95b8fdca492306b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:08.712906 9450 raft_consensus.cc:1261] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Refusing update from remote peer 274bb23d20a44efe95b8fdca492306b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:08.713352 9612 consensus_queue.cc:1035] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:08.713552 9617 consensus_queue.cc:1035] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:08.730546 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:08.727039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:41839 (local address 127.7.24.193:33381)
0620 14:53:08.727153 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:08.727159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:08.727373 (+ 214us) server_negotiation.cc:407] Connection header received
0620 14:53:08.727505 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:08.727507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:08.727562 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:08.727633 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:08.728004 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.728495 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.729312 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.729668 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.729758 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:08.730049 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:08.730131 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:08.730208 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:08.730312 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 14:53:08.730384 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:53:08.730585 9628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:08.726941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33381 (local address 127.7.24.195:41839)
0620 14:53:08.727295 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:08.727306 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:08.727387 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:08.727735 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:08.727737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:08.727763 (+ 26us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 14:53:08.727884 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:08.727891 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.728582 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.728585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:08.729196 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:08.729204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.729785 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.729788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:08.729896 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:08.729928 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 14:53:08.730220 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 14:53:08.730228 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 14:53:08.730303 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 14:53:08.730389 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":101,"threads_started":1}
I20260620 14:53:08.752082 9011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:08.752261 9011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:08.754518 9011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33: Sending DeleteTablet for 3 replicas of tablet 5a6c20e004464c849d55164a213d887f
I20260620 14:53:08.755087 9150 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a6c20e004464c849d55164a213d887f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:08 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:58590
I20260620 14:53:08.755266 9596 tablet_replica.cc:330] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8: stopping tablet replica
I20260620 14:53:08.755391 9596 raft_consensus.cc:2229] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:08.755375 9290 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a6c20e004464c849d55164a213d887f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:08 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:58532
I20260620 14:53:08.755541 9595 tablet_replica.cc:330] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7: stopping tablet replica
I20260620 14:53:08.755638 9595 raft_consensus.cc:2229] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:08.755721 9596 raft_consensus.cc:2258] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:08.755764 9595 raft_consensus.cc:2258] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:08.756286 9418 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a6c20e004464c849d55164a213d887f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:08 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:33102
I20260620 14:53:08.756474 9594 tablet_replica.cc:330] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8: stopping tablet replica
I20260620 14:53:08.756480 9596 ts_tablet_manager.cc:1905] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:08.756536 9595 ts_tablet_manager.cc:1905] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:08.756574 9594 raft_consensus.cc:2229] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:08.756680 9594 raft_consensus.cc:2258] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:08.757136 9011 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:08.757527 9011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:08.758982 9595 ts_tablet_manager.cc:1918] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:08.758982 9596 ts_tablet_manager.cc:1918] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:08.759058 9595 log.cc:1196] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/5a6c20e004464c849d55164a213d887f
I20260620 14:53:08.759058 9596 log.cc:1196] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-0/wal/wals/5a6c20e004464c849d55164a213d887f
I20260620 14:53:08.759318 9595 ts_tablet_manager.cc:1939] T 5a6c20e004464c849d55164a213d887f P 52fe6429041c498db60532f938ea38e7: Deleting consensus metadata
I20260620 14:53:08.759318 9596 ts_tablet_manager.cc:1939] T 5a6c20e004464c849d55164a213d887f P 274bb23d20a44efe95b8fdca492306b8: Deleting consensus metadata
I20260620 14:53:08.759904 9594 ts_tablet_manager.cc:1905] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:08.762413 9594 ts_tablet_manager.cc:1918] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:08.762511 9594 log.cc:1196] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/5a6c20e004464c849d55164a213d887f
I20260620 14:53:08.762768 9594 ts_tablet_manager.cc:1939] T 5a6c20e004464c849d55164a213d887f P 6c68ae7817e040729aaa92be7337ddc8: Deleting consensus metadata
I20260620 14:53:08.763074 9290 tablet_service.cc:1467] Processing CreateTablet for tablet 6d6f692ce60444418e64b3e1b2649114 (DEFAULT_TABLE table=test-table [id=17e39861ae8142a4adeb752f9beed67c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:08.763314 9290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d6f692ce60444418e64b3e1b2649114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:08.764135 9418 tablet_service.cc:1467] Processing CreateTablet for tablet 6d6f692ce60444418e64b3e1b2649114 (DEFAULT_TABLE table=test-table [id=17e39861ae8142a4adeb752f9beed67c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:08.764364 9418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d6f692ce60444418e64b3e1b2649114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:08.765352 9607 tablet_bootstrap.cc:492] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7: Bootstrap starting.
I20260620 14:53:08.766104 9607 tablet_bootstrap.cc:654] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:08.766597 9605 tablet_bootstrap.cc:492] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8: Bootstrap starting.
I20260620 14:53:08.767180 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 6d6f692ce60444418e64b3e1b2649114 (DEFAULT_TABLE table=test-table [id=17e39861ae8142a4adeb752f9beed67c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:08.767328 9605 tablet_bootstrap.cc:654] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:08.767412 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d6f692ce60444418e64b3e1b2649114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:08.767756 8999 catalog_manager.cc:4841] TS 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195:43839): tablet 5a6c20e004464c849d55164a213d887f (table test-table [id=1277553096444f8bac08870972599113]) successfully deleted
I20260620 14:53:08.768757 9607 tablet_bootstrap.cc:492] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7: No bootstrap required, opened a new log
I20260620 14:53:08.768824 9607 ts_tablet_manager.cc:1397] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 14:53:08.769176 9607 raft_consensus.cc:348] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.769280 9607 raft_consensus.cc:374] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:08.769308 9607 raft_consensus.cc:729] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52fe6429041c498db60532f938ea38e7, State: Initialized, Role: FOLLOWER
I20260620 14:53:08.769423 9607 consensus_queue.cc:260] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.769523 9606 tablet_bootstrap.cc:492] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8: Bootstrap starting.
I20260620 14:53:08.769644 9607 ts_tablet_manager.cc:1428] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 14:53:08.770409 9606 tablet_bootstrap.cc:654] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:08.770776 8997 catalog_manager.cc:4841] TS 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381): tablet 5a6c20e004464c849d55164a213d887f (table test-table [id=1277553096444f8bac08870972599113]) successfully deleted
I20260620 14:53:08.771524 8998 catalog_manager.cc:4841] TS 52fe6429041c498db60532f938ea38e7 (127.7.24.194:38547): tablet 5a6c20e004464c849d55164a213d887f (table test-table [id=1277553096444f8bac08870972599113]) successfully deleted
I20260620 14:53:08.771976 9606 tablet_bootstrap.cc:492] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8: No bootstrap required, opened a new log
I20260620 14:53:08.772066 9606 ts_tablet_manager.cc:1397] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:08.772421 9605 tablet_bootstrap.cc:492] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8: No bootstrap required, opened a new log
I20260620 14:53:08.772497 9605 ts_tablet_manager.cc:1397] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260620 14:53:08.772440 9606 raft_consensus.cc:348] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.772555 9606 raft_consensus.cc:374] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:08.772585 9606 raft_consensus.cc:729] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 274bb23d20a44efe95b8fdca492306b8, State: Initialized, Role: FOLLOWER
I20260620 14:53:08.772687 9606 consensus_queue.cc:260] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.772948 9606 ts_tablet_manager.cc:1428] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:08.772879 9605 raft_consensus.cc:348] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.772992 9605 raft_consensus.cc:374] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:08.773025 9605 raft_consensus.cc:729] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c68ae7817e040729aaa92be7337ddc8, State: Initialized, Role: FOLLOWER
I20260620 14:53:08.773164 9605 consensus_queue.cc:260] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.773406 9605 ts_tablet_manager.cc:1428] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:08.864810 9611 raft_consensus.cc:482] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:08.864962 9611 raft_consensus.cc:504] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.865499 9611 leader_election.cc:288] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381), 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195:43839)
I20260620 14:53:08.865995 9450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6f692ce60444418e64b3e1b2649114" candidate_uuid: "52fe6429041c498db60532f938ea38e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c68ae7817e040729aaa92be7337ddc8" is_pre_election: true
I20260620 14:53:08.866026 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6f692ce60444418e64b3e1b2649114" candidate_uuid: "52fe6429041c498db60532f938ea38e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274bb23d20a44efe95b8fdca492306b8" is_pre_election: true
I20260620 14:53:08.866179 9450 raft_consensus.cc:2454] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52fe6429041c498db60532f938ea38e7 in term 0.
I20260620 14:53:08.866187 9170 raft_consensus.cc:2454] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52fe6429041c498db60532f938ea38e7 in term 0.
I20260620 14:53:08.866537 9243 leader_election.cc:302] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52fe6429041c498db60532f938ea38e7, 6c68ae7817e040729aaa92be7337ddc8; no voters:
I20260620 14:53:08.866732 9611 raft_consensus.cc:2781] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:08.866801 9611 raft_consensus.cc:482] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:08.866833 9611 raft_consensus.cc:3037] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:08.867871 9611 raft_consensus.cc:504] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.868234 9611 leader_election.cc:288] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [CANDIDATE]: Term 1 election: Requested vote from peers 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381), 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195:43839)
I20260620 14:53:08.868574 9450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6f692ce60444418e64b3e1b2649114" candidate_uuid: "52fe6429041c498db60532f938ea38e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c68ae7817e040729aaa92be7337ddc8"
I20260620 14:53:08.868574 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d6f692ce60444418e64b3e1b2649114" candidate_uuid: "52fe6429041c498db60532f938ea38e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "274bb23d20a44efe95b8fdca492306b8"
I20260620 14:53:08.868695 9170 raft_consensus.cc:3037] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:08.868695 9450 raft_consensus.cc:3037] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:08.869777 9170 raft_consensus.cc:2454] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52fe6429041c498db60532f938ea38e7 in term 1.
I20260620 14:53:08.869777 9450 raft_consensus.cc:2454] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52fe6429041c498db60532f938ea38e7 in term 1.
I20260620 14:53:08.870047 9241 leader_election.cc:302] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 274bb23d20a44efe95b8fdca492306b8, 52fe6429041c498db60532f938ea38e7; no voters:
I20260620 14:53:08.870239 9611 raft_consensus.cc:2781] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:08.870463 9611 raft_consensus.cc:686] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 1 LEADER]: Becoming Leader. State: Replica: 52fe6429041c498db60532f938ea38e7, State: Running, Role: LEADER
I20260620 14:53:08.870635 9611 consensus_queue.cc:237] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } }
I20260620 14:53:08.872587 9011 catalog_manager.cc:5495] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52fe6429041c498db60532f938ea38e7 (127.7.24.194). New cstate: current_term: 1 leader_uuid: "52fe6429041c498db60532f938ea38e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52fe6429041c498db60532f938ea38e7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 } health_report { overall_health: UNKNOWN } } }
I20260620 14:53:08.909706 9614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:08.906258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58552 (local address 127.7.24.194:38547)
0620 14:53:08.906367 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:08.906371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:08.906412 (+ 41us) server_negotiation.cc:407] Connection header received
0620 14:53:08.906587 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:08.906590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:08.906639 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:08.906718 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:08.906970 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.907762 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.908537 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:08.908885 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:08.908991 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:08.909262 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:08.909341 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:08.909413 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:08.909487 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 14:53:08.909541 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 14:53:08.916535 9170 raft_consensus.cc:1261] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8 [term 1 FOLLOWER]: Refusing update from remote peer 52fe6429041c498db60532f938ea38e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:08.916605 9450 raft_consensus.cc:1261] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Refusing update from remote peer 52fe6429041c498db60532f938ea38e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:08.916986 9611 consensus_queue.cc:1035] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "274bb23d20a44efe95b8fdca492306b8" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:08.917264 9633 consensus_queue.cc:1035] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c68ae7817e040729aaa92be7337ddc8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 14:53:08.931948 9579 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 14:53:10.494236 9013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:10.494405 9013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:10.497077 9013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4245a551ded442dfb9405268d5f4fd33: Sending DeleteTablet for 3 replicas of tablet 6d6f692ce60444418e64b3e1b2649114
I20260620 14:53:10.497637 9290 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d6f692ce60444418e64b3e1b2649114 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:10 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:58532
I20260620 14:53:10.497709 9418 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d6f692ce60444418e64b3e1b2649114 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:10 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:33102
I20260620 14:53:10.497754 9150 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d6f692ce60444418e64b3e1b2649114 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:10 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:58590
I20260620 14:53:10.498086 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 9085
I20260620 14:53:10.498123 9664 tablet_replica.cc:330] T 6d6f692ce60444418e64b3e1b2649114 P 274bb23d20a44efe95b8fdca492306b8: stopping tablet replica
I20260620 14:53:10.498865 9662 tablet_replica.cc:330] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7: stopping tablet replica
I20260620 14:53:10.498984 9662 raft_consensus.cc:2229] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:10.499145 9662 raft_consensus.cc:2258] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:10.499373 9663 tablet_replica.cc:330] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8: stopping tablet replica
I20260620 14:53:10.499518 9663 raft_consensus.cc:2229] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:10.499642 9663 raft_consensus.cc:2258] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:10.499760 9662 ts_tablet_manager.cc:1905] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:10.500265 9663 ts_tablet_manager.cc:1905] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:10.502776 9662 ts_tablet_manager.cc:1918] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 14:53:10.502868 9662 log.cc:1196] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/6d6f692ce60444418e64b3e1b2649114
I20260620 14:53:10.503075 9663 ts_tablet_manager.cc:1918] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 14:53:10.503157 9663 log.cc:1196] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/6d6f692ce60444418e64b3e1b2649114
I20260620 14:53:10.503257 9662 ts_tablet_manager.cc:1939] T 6d6f692ce60444418e64b3e1b2649114 P 52fe6429041c498db60532f938ea38e7: Deleting consensus metadata
I20260620 14:53:10.503437 9663 ts_tablet_manager.cc:1939] T 6d6f692ce60444418e64b3e1b2649114 P 6c68ae7817e040729aaa92be7337ddc8: Deleting consensus metadata
I20260620 14:53:10.504074 8998 catalog_manager.cc:4841] TS 52fe6429041c498db60532f938ea38e7 (127.7.24.194:38547): tablet 6d6f692ce60444418e64b3e1b2649114 (table test-table [id=17e39861ae8142a4adeb752f9beed67c]) successfully deleted
I20260620 14:53:10.504089 8999 catalog_manager.cc:4841] TS 6c68ae7817e040729aaa92be7337ddc8 (127.7.24.195:43839): tablet 6d6f692ce60444418e64b3e1b2649114 (table test-table [id=17e39861ae8142a4adeb752f9beed67c]) successfully deleted
I20260620 14:53:10.507716 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 9225
W20260620 14:53:10.507669 8997 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:33381 (error 108)
I20260620 14:53:10.509742 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:10.508741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33381 (local address 127.0.0.1:58602)
0620 14:53:10.509517 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:10.509604 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":82,"threads_started":1}
W20260620 14:53:10.509945 8997 catalog_manager.cc:4584] TS 274bb23d20a44efe95b8fdca492306b8 (127.7.24.193:33381): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6d6f692ce60444418e64b3e1b2649114: Network error: Client connection negotiation failed: client connection to 127.7.24.193:33381: connect: Connection refused (error 111)
I20260620 14:53:10.515058 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 9365
I20260620 14:53:10.522199 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 8981
2026-06-20T14:53:10Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6143 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:10 dist-test-slave-sms9 krb5kdc[9669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:10 dist-test-slave-sms9 krb5kdc[9669](info): set up 2 sockets
Jun 20 14:53:10 dist-test-slave-sms9 krb5kdc[9669](info): commencing operation
krb5kdc: starting...
W20260620 14:53:11.586599 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:11 dist-test-slave-sms9 krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:11.726042 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:44173
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:44173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:11.854749 9682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:11.855086 9682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:11.855163 9682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:11.859287 9682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:11.859390 9682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:11.859428 9682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:11.859465 9682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:11.859503 9682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:11.865053 9682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:44173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:44173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:11.866253 9682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:11.867408 9682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:11.877379 9687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:11.877379 9690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:11.877439 9682 server_base.cc:1034] running on GCE node
W20260620 14:53:11.877372 9688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:11.878577 9682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:11.879810 9682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:11.880983 9682 hybrid_clock.cc:648] HybridClock initialized: now 1781967191880951 us; error 44 us; skew 500 ppm
Jun 20 14:53:11 dist-test-slave-sms9 krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967191, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:11.884028 9682 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:11.884953 9682 webserver.cc:460] Webserver started at http://127.7.24.254:38411/ using document root <none> and password file <none>
I20260620 14:53:11.885499 9682 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:11.885556 9682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:11.885931 9682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:11.888258 9682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "6dff0baa1e1f416d83842f2589da1a3f"
format_stamp: "Formatted at 2026-06-20 14:53:11 on dist-test-slave-sms9"
server_key: "f0c5cf8b3da06f5ea938b0e5d47cac48"
server_key_iv: "99e89d58e85ea889156bc9ca348894dc"
server_key_version: "encryptionkey@0"
I20260620 14:53:11.888808 9682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "6dff0baa1e1f416d83842f2589da1a3f"
format_stamp: "Formatted at 2026-06-20 14:53:11 on dist-test-slave-sms9"
server_key: "f0c5cf8b3da06f5ea938b0e5d47cac48"
server_key_iv: "99e89d58e85ea889156bc9ca348894dc"
server_key_version: "encryptionkey@0"
I20260620 14:53:11.892611 9682 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:53:11.895025 9697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:11.896317 9682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:11.896420 9682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "6dff0baa1e1f416d83842f2589da1a3f"
format_stamp: "Formatted at 2026-06-20 14:53:11 on dist-test-slave-sms9"
server_key: "f0c5cf8b3da06f5ea938b0e5d47cac48"
server_key_iv: "99e89d58e85ea889156bc9ca348894dc"
server_key_version: "encryptionkey@0"
I20260620 14:53:11.896559 9682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:11.911887 9682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:11.915211 9682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:11.915413 9682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:11.922706 9682 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:44173
I20260620 14:53:11.922729 9749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:44173 every 8 connection(s)
I20260620 14:53:11.923765 9682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:11.924113 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 9682
I20260620 14:53:11.924244 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:11.924554 7267 external_mini_cluster.cc:1428] Setting key daefe5a1178a457483129acffe568662
I20260620 14:53:11.926981 9750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 20 14:53:11 dist-test-slave-sms9 krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:11.933792 9750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f: Bootstrap starting.
I20260620 14:53:11.936111 9750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:11.936837 9750 log.cc:824] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:11.938836 9750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f: No bootstrap required, opened a new log
I20260620 14:53:11.939517 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:11.925908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.7.24.254:44173)
0620 14:53:11.926321 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:11.926341 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:11.926384 (+ 43us) server_negotiation.cc:407] Connection header received
0620 14:53:11.927144 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:11.927165 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:11.927503 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:11.927829 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:11.928717 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:11.929777 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:11.930975 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:11.931428 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:11.931547 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:11.934430 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:11.934435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:11.934448 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:11.934521 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:11.936388 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:11.936844 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:11.936850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:11.936862 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:11.937002 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:11.937293 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:11.937297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:11.937299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:11.937696 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:11.937906 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:11.938176 (+ 270us) server_negotiation.cc:299] Negotiation successful
0620 14:53:11.938523 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":156,"threads_started":1}
I20260620 14:53:11.941851 9750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dff0baa1e1f416d83842f2589da1a3f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44173 } }
I20260620 14:53:11.942092 9750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:11.942176 9750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dff0baa1e1f416d83842f2589da1a3f, State: Initialized, Role: FOLLOWER
I20260620 14:53:11.942637 9750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dff0baa1e1f416d83842f2589da1a3f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44173 } }
I20260620 14:53:11.942795 9750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:11.942871 9750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:11.942992 9750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:11.943871 9750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dff0baa1e1f416d83842f2589da1a3f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44173 } }
I20260620 14:53:11.944207 9750 leader_election.cc:302] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dff0baa1e1f416d83842f2589da1a3f; no voters:
I20260620 14:53:11.944504 9750 leader_election.cc:288] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:11.944659 9755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:11.944960 9755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [term 1 LEADER]: Becoming Leader. State: Replica: 6dff0baa1e1f416d83842f2589da1a3f, State: Running, Role: LEADER
I20260620 14:53:11.945277 9755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dff0baa1e1f416d83842f2589da1a3f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44173 } }
I20260620 14:53:11.945731 9750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:11.947072 9757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dff0baa1e1f416d83842f2589da1a3f. Latest consensus state: current_term: 1 leader_uuid: "6dff0baa1e1f416d83842f2589da1a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dff0baa1e1f416d83842f2589da1a3f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44173 } } }
I20260620 14:53:11.947067 9756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dff0baa1e1f416d83842f2589da1a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dff0baa1e1f416d83842f2589da1a3f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44173 } } }
I20260620 14:53:11.947249 9756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:11.947249 9757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:11.947796 9766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:11.950974 9766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:11.956552 9766 catalog_manager.cc:1347] Generated new cluster ID: 6e3051df56db4653a36a7df909ff352d
I20260620 14:53:11.956645 9766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:11.982016 9766 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:11.983501 9766 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:11.993146 9766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dff0baa1e1f416d83842f2589da1a3f: Generated new TSK 0
I20260620 14:53:11.994004 9766 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:53:12.034090 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44173
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:12.150264 9776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:12.150588 9776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:12.150658 9776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:12.154654 9776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:12.154707 9776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:12.154791 9776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:12.160126 9776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:12.161194 9776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:12.162240 9776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:12.172329 9781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:12.172415 9782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:12.172333 9784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:12.172537 9776 server_base.cc:1034] running on GCE node
I20260620 14:53:12.173110 9776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:12.173806 9776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:12.175036 9776 hybrid_clock.cc:648] HybridClock initialized: now 1781967192174971 us; error 47 us; skew 500 ppm
Jun 20 14:53:12 dist-test-slave-sms9 krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:12.177897 9776 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:53:12.178809 9776 webserver.cc:460] Webserver started at http://127.7.24.193:39005/ using document root <none> and password file <none>
I20260620 14:53:12.179363 9776 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:12.179450 9776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:12.179672 9776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:12.181725 9776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "55e3678b708f4e18a978f955c8553b4e"
format_stamp: "Formatted at 2026-06-20 14:53:12 on dist-test-slave-sms9"
server_key: "f2bfa4f54b0b305e4e8af6f5fe08b09e"
server_key_iv: "7a0c1927ea20b75529c14684cc760f17"
server_key_version: "encryptionkey@0"
I20260620 14:53:12.182287 9776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "55e3678b708f4e18a978f955c8553b4e"
format_stamp: "Formatted at 2026-06-20 14:53:12 on dist-test-slave-sms9"
server_key: "f2bfa4f54b0b305e4e8af6f5fe08b09e"
server_key_iv: "7a0c1927ea20b75529c14684cc760f17"
server_key_version: "encryptionkey@0"
I20260620 14:53:12.185957 9776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:12.188563 9791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:12.189883 9776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:12.190026 9776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "55e3678b708f4e18a978f955c8553b4e"
format_stamp: "Formatted at 2026-06-20 14:53:12 on dist-test-slave-sms9"
server_key: "f2bfa4f54b0b305e4e8af6f5fe08b09e"
server_key_iv: "7a0c1927ea20b75529c14684cc760f17"
server_key_version: "encryptionkey@0"
I20260620 14:53:12.190158 9776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:12.222559 9776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:12.226609 9776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:12.226822 9776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:12.227473 9776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:12.228336 9776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:12.228407 9776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:12.228474 9776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:12.228523 9776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:12.237284 9776 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:45841
I20260620 14:53:12.237318 9904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:45841 every 8 connection(s)
I20260620 14:53:12.238212 9776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:12.241094 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 9776
I20260620 14:53:12.241231 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:12.241580 7267 external_mini_cluster.cc:1428] Setting key d8958edf61211a7464a0dcdfd4229ab4
Jun 20 14:53:12 dist-test-slave-sms9 krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
I20260620 14:53:12.254994 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:12.240115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:59051 (local address 127.7.24.254:44173)
0620 14:53:12.240252 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:12.240256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:12.241228 (+ 972us) server_negotiation.cc:407] Connection header received
0620 14:53:12.242987 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:12.242991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:12.243051 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:12.243139 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:12.244915 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.245770 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.247237 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.247724 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.247855 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:12.250893 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:12.250898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:12.250901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:12.250937 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:12.252541 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:12.253098 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:12.253101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:12.253102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:12.253186 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:12.253516 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:12.253518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:12.253519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:12.253753 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:12.253835 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:12.254514 (+ 679us) server_negotiation.cc:299] Negotiation successful
0620 14:53:12.254740 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:53:12.256073 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:12.240412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:44173 (local address 127.7.24.193:59051)
0620 14:53:12.241008 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:12.241085 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:12.242763 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:12.243357 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:12.243370 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:12.243934 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:12.244715 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:12.244739 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.245918 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.245921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:12.247094 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:12.247104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.247889 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.247892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:12.248046 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:12.248848 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:12.248866 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:12.250695 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:12.252662 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:12.252667 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:12.252681 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:12.252994 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:12.253280 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:12.253283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:12.253284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:12.253425 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:12.253857 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:12.253866 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:12.254152 (+ 286us) client_negotiation.cc:769] Sending connection context
0620 14:53:12.254391 (+ 239us) client_negotiation.cc:240] Negotiation successful
0620 14:53:12.254840 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":128,"threads_started":1}
I20260620 14:53:12.257395 9905 heartbeater.cc:347] Connected to a master server at 127.7.24.254:44173
I20260620 14:53:12.257683 9905 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:12.258217 9905 heartbeater.cc:511] Master 127.7.24.254:44173 requested a full tablet report, sending...
I20260620 14:53:12.259853 9714 ts_manager.cc:194] Registered new tserver with Master: 55e3678b708f4e18a978f955c8553b4e (127.7.24.193:45841)
I20260620 14:53:12.261041 9714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:59051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:53:12.271674 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44173
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:12.383332 9910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:12.383630 9910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:12.383716 9910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:12.387638 9910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:12.387732 9910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:12.387822 9910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:12.393271 9910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:12.394330 9910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:12.395442 9910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:12.405485 9918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:12.405464 9915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:12.405660 9910 server_base.cc:1034] running on GCE node
W20260620 14:53:12.405468 9916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:12.406177 9910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:12.406879 9910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:12.408092 9910 hybrid_clock.cc:648] HybridClock initialized: now 1781967192408050 us; error 53 us; skew 500 ppm
Jun 20 14:53:12 dist-test-slave-sms9 krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:12.410889 9910 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:53:12.411695 9910 webserver.cc:460] Webserver started at http://127.7.24.194:33101/ using document root <none> and password file <none>
I20260620 14:53:12.412209 9910 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:12.412266 9910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:12.412428 9910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:12.414371 9910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "464c1b3671194584b2d4a27271c8595c"
format_stamp: "Formatted at 2026-06-20 14:53:12 on dist-test-slave-sms9"
server_key: "fe756548010f57402d5d9f3e1a268b12"
server_key_iv: "ffb1655afd48fe6b84ab1fcb29f9b8b8"
server_key_version: "encryptionkey@0"
I20260620 14:53:12.414925 9910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "464c1b3671194584b2d4a27271c8595c"
format_stamp: "Formatted at 2026-06-20 14:53:12 on dist-test-slave-sms9"
server_key: "fe756548010f57402d5d9f3e1a268b12"
server_key_iv: "ffb1655afd48fe6b84ab1fcb29f9b8b8"
server_key_version: "encryptionkey@0"
I20260620 14:53:12.418531 9910 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:12.420825 9925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:12.422066 9910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:12.422170 9910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "464c1b3671194584b2d4a27271c8595c"
format_stamp: "Formatted at 2026-06-20 14:53:12 on dist-test-slave-sms9"
server_key: "fe756548010f57402d5d9f3e1a268b12"
server_key_iv: "ffb1655afd48fe6b84ab1fcb29f9b8b8"
server_key_version: "encryptionkey@0"
I20260620 14:53:12.422256 9910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:12.435051 9910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:12.438439 9910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:12.438719 9910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:12.439303 9910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:12.440109 9910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:12.440181 9910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:12.440253 9910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:12.440307 9910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:12.450847 9910 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:39185
I20260620 14:53:12.450899 10038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:39185 every 8 connection(s)
I20260620 14:53:12.451756 9910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:12.458163 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 9910
I20260620 14:53:12.458271 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:12.458580 7267 external_mini_cluster.cc:1428] Setting key d45f4f622b257d6a0777b514300ca138
Jun 20 14:53:12 dist-test-slave-sms9 krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:12.468446 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:12.453829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:49179 (local address 127.7.24.254:44173)
0620 14:53:12.453997 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:12.454003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:12.455095 (+ 1092us) server_negotiation.cc:407] Connection header received
0620 14:53:12.456381 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:12.456384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:12.456434 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:12.456512 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:12.458063 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.458907 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.460349 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.460769 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.460893 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:12.463797 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:12.463802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:12.463807 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:12.463841 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:12.465755 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:12.466480 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:12.466485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:12.466487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:12.466591 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:12.467025 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:12.467031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:12.467032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:12.467286 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:12.467399 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:12.468074 (+ 675us) server_negotiation.cc:299] Negotiation successful
0620 14:53:12.468225 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:53:12.469518 10041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:12.454140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:44173 (local address 127.7.24.194:49179)
0620 14:53:12.454920 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:12.454978 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:12.456135 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:12.456691 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:12.456700 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:12.457191 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:12.457873 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:12.457895 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.459056 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.459060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:12.460217 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:12.460229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.460951 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.460958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:12.461206 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:12.461964 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:12.461981 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:12.463598 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:12.465908 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:12.465917 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:12.465935 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:12.466336 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:12.466711 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:12.466714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:12.466716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:12.466914 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:12.467416 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:12.467423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:12.467779 (+ 356us) client_negotiation.cc:769] Sending connection context
0620 14:53:12.468010 (+ 231us) client_negotiation.cc:240] Negotiation successful
0620 14:53:12.468383 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":102,"threads_started":1}
I20260620 14:53:12.470889 10039 heartbeater.cc:347] Connected to a master server at 127.7.24.254:44173
I20260620 14:53:12.471238 10039 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
I20260620 14:53:12.471895 10039 heartbeater.cc:511] Master 127.7.24.254:44173 requested a full tablet report, sending...
I20260620 14:53:12.473124 9714 ts_manager.cc:194] Registered new tserver with Master: 464c1b3671194584b2d4a27271c8595c (127.7.24.194:39185)
I20260620 14:53:12.473970 9714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:49179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:53:12.490319 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44173
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:46321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:12.601045 10044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:12.601361 10044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:12.601435 10044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:12.605636 10044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:12.605753 10044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:12.605851 10044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:12.611459 10044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:46321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:12.612603 10044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:12.613680 10044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:12.623327 10050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:12.623348 10049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:12.623636 10044 server_base.cc:1034] running on GCE node
W20260620 14:53:12.623348 10052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:12.624097 10044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:12.624744 10044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:12.625914 10044 hybrid_clock.cc:648] HybridClock initialized: now 1781967192625871 us; error 55 us; skew 500 ppm
Jun 20 14:53:12 dist-test-slave-sms9 krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:12.628888 10044 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:53:12.629773 10044 webserver.cc:460] Webserver started at http://127.7.24.195:39711/ using document root <none> and password file <none>
I20260620 14:53:12.630335 10044 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:12.630394 10044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:12.630630 10044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:12.632639 10044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "2b8652c0bf704cf8a5460c6de7426e64"
format_stamp: "Formatted at 2026-06-20 14:53:12 on dist-test-slave-sms9"
server_key: "be996d4718fd9fe89d63e1f3b2d38197"
server_key_iv: "1e6b6d493f064af25926d3844f083650"
server_key_version: "encryptionkey@0"
I20260620 14:53:12.633199 10044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "2b8652c0bf704cf8a5460c6de7426e64"
format_stamp: "Formatted at 2026-06-20 14:53:12 on dist-test-slave-sms9"
server_key: "be996d4718fd9fe89d63e1f3b2d38197"
server_key_iv: "1e6b6d493f064af25926d3844f083650"
server_key_version: "encryptionkey@0"
I20260620 14:53:12.636970 10044 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 14:53:12.639460 10059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:12.640637 10044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:12.640760 10044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "2b8652c0bf704cf8a5460c6de7426e64"
format_stamp: "Formatted at 2026-06-20 14:53:12 on dist-test-slave-sms9"
server_key: "be996d4718fd9fe89d63e1f3b2d38197"
server_key_iv: "1e6b6d493f064af25926d3844f083650"
server_key_version: "encryptionkey@0"
I20260620 14:53:12.640870 10044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:12.665287 10044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:12.668771 10044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:12.668980 10044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:12.669611 10044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:12.670559 10044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:12.670608 10044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:12.670670 10044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:12.670702 10044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:12.679842 10044 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:43133
I20260620 14:53:12.679865 10172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:43133 every 8 connection(s)
I20260620 14:53:12.680742 10044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:12.687510 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 10044
I20260620 14:53:12.687608 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:12.687912 7267 external_mini_cluster.cc:1428] Setting key 94b3476d32d7b5c2b749cbd998f9abbd
Jun 20 14:53:12 dist-test-slave-sms9 krb5kdc[9669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:12.694794 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:12.682655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:33171 (local address 127.7.24.254:44173)
0620 14:53:12.682810 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:12.682814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:12.683918 (+ 1104us) server_negotiation.cc:407] Connection header received
0620 14:53:12.684767 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:12.684770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:12.684818 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:12.684899 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:12.686107 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.686825 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.688217 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.688471 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.688558 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:12.691276 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:12.691280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:12.691283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:12.691307 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:12.692776 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:12.693240 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:12.693243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:12.693244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:12.693365 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:12.693699 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:12.693703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:12.693705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:12.693897 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:12.693989 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:12.694539 (+ 550us) server_negotiation.cc:299] Negotiation successful
0620 14:53:12.694667 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 14:53:12.695569 10175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:12.683096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:44173 (local address 127.7.24.195:33171)
0620 14:53:12.683740 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:12.683813 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:12.684552 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:12.685053 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:12.685059 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:12.685412 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:12.685933 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:12.685946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.686959 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.686962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:12.688101 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:12.688111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:12.688607 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:12.688614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:12.688787 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:12.689493 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:12.689509 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:12.691131 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:12.692886 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:12.692891 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:12.692902 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:12.693145 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:12.693454 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:12.693456 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:12.693457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:12.693583 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:12.693999 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:12.694004 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:12.694228 (+ 224us) client_negotiation.cc:769] Sending connection context
0620 14:53:12.694485 (+ 257us) client_negotiation.cc:240] Negotiation successful
0620 14:53:12.694762 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":115,"threads_started":1}
I20260620 14:53:12.697062 10173 heartbeater.cc:347] Connected to a master server at 127.7.24.254:44173
I20260620 14:53:12.697281 10173 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:12.697726 10173 heartbeater.cc:511] Master 127.7.24.254:44173 requested a full tablet report, sending...
I20260620 14:53:12.698715 9714 ts_manager.cc:194] Registered new tserver with Master: 2b8652c0bf704cf8a5460c6de7426e64 (127.7.24.195:43133)
I20260620 14:53:12.699368 9714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:33171
I20260620 14:53:12.702117 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 14:53:12.715175 9753 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:12.713773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46624 (local address 127.7.24.254:44173)
0620 14:53:12.713925 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:12.713929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:12.714114 (+ 185us) server_negotiation.cc:407] Connection header received
0620 14:53:12.714204 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:12.714207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:12.714252 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:12.714332 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:12.715052 (+ 720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46624: BlockingRecv error: recv got EOF from 127.0.0.1:46624 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 14:53:12.716411 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 9776
I20260620 14:53:12.721879 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 9910
I20260620 14:53:12.727672 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 10044
I20260620 14:53:12.733155 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 9682
2026-06-20T14:53:12Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2210 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:12 dist-test-slave-sms9 krb5kdc[10185](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:12 dist-test-slave-sms9 krb5kdc[10185](info): set up 2 sockets
Jun 20 14:53:12 dist-test-slave-sms9 krb5kdc[10185](info): commencing operation
krb5kdc: starting...
W20260620 14:53:13.778440 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:13 dist-test-slave-sms9 krb5kdc[10185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:13.902168 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:34551
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:34551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:14.011229 10198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:14.011566 10198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:14.011651 10198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:14.015862 10198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:14.015949 10198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:14.015985 10198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:14.016022 10198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:14.016059 10198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:14.021325 10198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:34551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:34551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:14.022717 10198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:14.023819 10198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:53:14.033298 10198 server_base.cc:1034] running on GCE node
W20260620 14:53:14.033252 10204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:14.033203 10203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:14.033208 10206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:14.034101 10198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:14.035300 10198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:14.036495 10198 hybrid_clock.cc:648] HybridClock initialized: now 1781967194036478 us; error 31 us; skew 500 ppm
Jun 20 14:53:14 dist-test-slave-sms9 krb5kdc[10185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967194, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:14.044152 10198 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:14.045130 10198 webserver.cc:460] Webserver started at http://127.7.24.254:36407/ using document root <none> and password file <none>
I20260620 14:53:14.045672 10198 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:14.045732 10198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:14.045903 10198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:14.048004 10198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "4f4b680a1fe04dbc9a4182aa7e6b1c29"
format_stamp: "Formatted at 2026-06-20 14:53:14 on dist-test-slave-sms9"
server_key: "e80f0aad8809004deea8fa1ef0dc3ad2"
server_key_iv: "7556a3f1ac454da1802ef013d313ea9f"
server_key_version: "encryptionkey@0"
I20260620 14:53:14.048539 10198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "4f4b680a1fe04dbc9a4182aa7e6b1c29"
format_stamp: "Formatted at 2026-06-20 14:53:14 on dist-test-slave-sms9"
server_key: "e80f0aad8809004deea8fa1ef0dc3ad2"
server_key_iv: "7556a3f1ac454da1802ef013d313ea9f"
server_key_version: "encryptionkey@0"
I20260620 14:53:14.052327 10198 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260620 14:53:14.054859 10213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:14.056334 10198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 14:53:14.056483 10198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "4f4b680a1fe04dbc9a4182aa7e6b1c29"
format_stamp: "Formatted at 2026-06-20 14:53:14 on dist-test-slave-sms9"
server_key: "e80f0aad8809004deea8fa1ef0dc3ad2"
server_key_iv: "7556a3f1ac454da1802ef013d313ea9f"
server_key_version: "encryptionkey@0"
I20260620 14:53:14.056599 10198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:14.075269 10198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:14.078830 10198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:14.079039 10198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:14.086413 10198 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:34551
I20260620 14:53:14.086724 10265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:34551 every 8 connection(s)
I20260620 14:53:14.087445 10198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:14.088439 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 10198
I20260620 14:53:14.088560 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:14.088850 7267 external_mini_cluster.cc:1428] Setting key c2252087a2232a67c482d034daf610f8
I20260620 14:53:14.090799 10266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:14.097261 10266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29: Bootstrap starting.
Jun 20 14:53:14 dist-test-slave-sms9 krb5kdc[10185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:14.099985 10266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:14.100835 10266 log.cc:824] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:14.103188 10266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29: No bootstrap required, opened a new log
I20260620 14:53:14.103483 10269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:14.090118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44498 (local address 127.7.24.254:34551)
0620 14:53:14.090559 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:14.090576 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:14.090612 (+ 36us) server_negotiation.cc:407] Connection header received
0620 14:53:14.091300 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:14.091318 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:14.091631 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:14.091909 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:14.092841 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:14.093914 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:14.095043 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:14.095507 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:14.095643 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:14.098452 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:14.098482 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:14.098499 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:14.098535 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:14.100485 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:14.100914 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:14.100919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:14.100926 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:14.101056 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:14.101345 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:14.101348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:14.101350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:14.101809 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:14.101989 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:14.102251 (+ 262us) server_negotiation.cc:299] Negotiation successful
0620 14:53:14.102576 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":150,"threads_started":1}
I20260620 14:53:14.106407 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:53:14.107259 10266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4b680a1fe04dbc9a4182aa7e6b1c29" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34551 } }
I20260620 14:53:14.107496 10266 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:14.107585 10266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f4b680a1fe04dbc9a4182aa7e6b1c29, State: Initialized, Role: FOLLOWER
I20260620 14:53:14.108003 10266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4b680a1fe04dbc9a4182aa7e6b1c29" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34551 } }
I20260620 14:53:14.108147 10266 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:14.108227 10266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:14.108330 10266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:14.109227 10266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4b680a1fe04dbc9a4182aa7e6b1c29" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34551 } }
I20260620 14:53:14.109555 10266 leader_election.cc:302] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f4b680a1fe04dbc9a4182aa7e6b1c29; no voters:
I20260620 14:53:14.109903 10266 leader_election.cc:288] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:14.109985 10272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:14.110163 10272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [term 1 LEADER]: Becoming Leader. State: Replica: 4f4b680a1fe04dbc9a4182aa7e6b1c29, State: Running, Role: LEADER
I20260620 14:53:14.110527 10272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4b680a1fe04dbc9a4182aa7e6b1c29" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34551 } }
I20260620 14:53:14.111164 10266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:14.111639 10274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f4b680a1fe04dbc9a4182aa7e6b1c29. Latest consensus state: current_term: 1 leader_uuid: "4f4b680a1fe04dbc9a4182aa7e6b1c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4b680a1fe04dbc9a4182aa7e6b1c29" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34551 } } }
I20260620 14:53:14.111806 10274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:14.112584 10273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f4b680a1fe04dbc9a4182aa7e6b1c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4b680a1fe04dbc9a4182aa7e6b1c29" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34551 } } }
I20260620 14:53:14.112764 10273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:14.113629 10278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:14.117470 10278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:14.123380 10278 catalog_manager.cc:1347] Generated new cluster ID: 5d04c3632b6449f59942d2dd1a64146c
I20260620 14:53:14.123446 10278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:14.125399 10269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:14.119909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44508 (local address 127.7.24.254:34551)
0620 14:53:14.120030 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:14.120035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:14.120318 (+ 283us) server_negotiation.cc:407] Connection header received
0620 14:53:14.120447 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:14.120452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:14.120520 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:14.120634 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:14.121223 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:14.122040 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:14.123218 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:14.123567 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:14.123670 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:14.124564 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:14.124568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:14.124570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:14.124973 (+ 403us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:14.125001 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:14.125017 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:14.125118 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:14.125201 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 14:53:14.125263 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 14:53:14.140532 10278 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:14.141502 10278 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:14.152433 10278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f4b680a1fe04dbc9a4182aa7e6b1c29: Generated new TSK 0
I20260620 14:53:14.153177 10278 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:53:14.164688 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 10198
2026-06-20T14:53:14Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1430 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:14 dist-test-slave-sms9 krb5kdc[10299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:14 dist-test-slave-sms9 krb5kdc[10299](info): set up 2 sockets
Jun 20 14:53:14 dist-test-slave-sms9 krb5kdc[10299](info): commencing operation
krb5kdc: starting...
W20260620 14:53:15.226684 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:15 dist-test-slave-sms9 krb5kdc[10299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:15.367254 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:34285
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:34285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:15.481899 10312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:15.482262 10312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:15.482331 10312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:15.486958 10312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:15.487078 10312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:15.487114 10312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:15.487156 10312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:15.487195 10312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:15.493001 10312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:34285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:34285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:15.494206 10312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:15.495348 10312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:15.507715 10318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:15.507722 10317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:15.507853 10312 server_base.cc:1034] running on GCE node
W20260620 14:53:15.508113 10320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:15.508898 10312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:15.510329 10312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:15.511524 10312 hybrid_clock.cc:648] HybridClock initialized: now 1781967195511507 us; error 39 us; skew 500 ppm
Jun 20 14:53:15 dist-test-slave-sms9 krb5kdc[10299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:15.515069 10312 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:15.516287 10312 webserver.cc:460] Webserver started at http://127.7.24.254:46511/ using document root <none> and password file <none>
I20260620 14:53:15.516966 10312 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:15.517042 10312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:15.517336 10312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:15.519748 10312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "408acd302c354bc9ac41b8c55a4da032"
format_stamp: "Formatted at 2026-06-20 14:53:15 on dist-test-slave-sms9"
server_key: "55bfe83fce074fd31885ebca27698738"
server_key_iv: "c52ff25477c9056353c8c2a49a9a3d30"
server_key_version: "encryptionkey@0"
I20260620 14:53:15.520404 10312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "408acd302c354bc9ac41b8c55a4da032"
format_stamp: "Formatted at 2026-06-20 14:53:15 on dist-test-slave-sms9"
server_key: "55bfe83fce074fd31885ebca27698738"
server_key_iv: "c52ff25477c9056353c8c2a49a9a3d30"
server_key_version: "encryptionkey@0"
I20260620 14:53:15.524930 10312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:53:15.527974 10327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:15.529592 10312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 14:53:15.529785 10312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "408acd302c354bc9ac41b8c55a4da032"
format_stamp: "Formatted at 2026-06-20 14:53:15 on dist-test-slave-sms9"
server_key: "55bfe83fce074fd31885ebca27698738"
server_key_iv: "c52ff25477c9056353c8c2a49a9a3d30"
server_key_version: "encryptionkey@0"
I20260620 14:53:15.529922 10312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:15.546782 10312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:15.550119 10312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:15.550352 10312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:15.558377 10312 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:34285
I20260620 14:53:15.558786 10379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:34285 every 8 connection(s)
I20260620 14:53:15.559465 10312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:15.563210 10380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:15.565541 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 10312
I20260620 14:53:15.565694 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:15.566053 7267 external_mini_cluster.cc:1428] Setting key 7f95c215e42d65f932afc1e00d43ad12
I20260620 14:53:15.571739 10380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032: Bootstrap starting.
I20260620 14:53:15.574906 10380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:15.576076 10380 log.cc:824] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032: Log is configured to *not* fsync() on all Append() calls
Jun 20 14:53:15 dist-test-slave-sms9 krb5kdc[10299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:15.578581 10380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032: No bootstrap required, opened a new log
I20260620 14:53:15.582679 10380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408acd302c354bc9ac41b8c55a4da032" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34285 } }
I20260620 14:53:15.582970 10380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:15.583052 10380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408acd302c354bc9ac41b8c55a4da032, State: Initialized, Role: FOLLOWER
I20260620 14:53:15.583485 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:15.568003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59296 (local address 127.7.24.254:34285)
0620 14:53:15.568515 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:15.568532 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:15.568575 (+ 43us) server_negotiation.cc:407] Connection header received
0620 14:53:15.569457 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:15.569478 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:15.569862 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:15.570188 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:15.571311 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:15.572502 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:15.573822 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:15.574356 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:15.574570 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:15.577887 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:15.577893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:15.577909 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:15.577940 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:15.580073 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:15.580615 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:15.580621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:15.580626 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:15.580752 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:15.581065 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:15.581071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:15.581073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:15.581524 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:15.581786 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:15.582123 (+ 337us) server_negotiation.cc:299] Negotiation successful
0620 14:53:15.582505 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":168,"threads_started":1}
I20260620 14:53:15.583678 10380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408acd302c354bc9ac41b8c55a4da032" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34285 } }
I20260620 14:53:15.583859 10380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:15.583930 10380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:15.584031 10380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:15.585340 10380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408acd302c354bc9ac41b8c55a4da032" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34285 } }
I20260620 14:53:15.585745 10380 leader_election.cc:302] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 408acd302c354bc9ac41b8c55a4da032; no voters:
I20260620 14:53:15.586159 10380 leader_election.cc:288] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:15.586340 10385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:15.586673 10385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [term 1 LEADER]: Becoming Leader. State: Replica: 408acd302c354bc9ac41b8c55a4da032, State: Running, Role: LEADER
I20260620 14:53:15.587069 10385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408acd302c354bc9ac41b8c55a4da032" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34285 } }
I20260620 14:53:15.587698 10380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:15.588956 10387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 408acd302c354bc9ac41b8c55a4da032. Latest consensus state: current_term: 1 leader_uuid: "408acd302c354bc9ac41b8c55a4da032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408acd302c354bc9ac41b8c55a4da032" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34285 } } }
I20260620 14:53:15.589104 10387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:15.589519 10394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:15.589438 10386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "408acd302c354bc9ac41b8c55a4da032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408acd302c354bc9ac41b8c55a4da032" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34285 } } }
I20260620 14:53:15.589682 10386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:15.593014 10394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:15.599848 10394 catalog_manager.cc:1347] Generated new cluster ID: 690be506bacf4c639351ef6c18b24333
I20260620 14:53:15.599944 10394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:15.609793 10394 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:15.611016 10394 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:15.622299 10394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 408acd302c354bc9ac41b8c55a4da032: Generated new TSK 0
I20260620 14:53:15.623143 10394 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:53:15.657866 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34285
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:15.792205 10406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:15.792474 10406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:15.792517 10406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:15.796552 10406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:15.796618 10406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:15.796707 10406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:15.802039 10406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:15.803208 10406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:15.804278 10406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:15.814720 10414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:15.814680 10411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:15.814605 10412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:15.814654 10406 server_base.cc:1034] running on GCE node
I20260620 14:53:15.815377 10406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:15.816097 10406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:15.817293 10406 hybrid_clock.cc:648] HybridClock initialized: now 1781967195817255 us; error 58 us; skew 500 ppm
Jun 20 14:53:15 dist-test-slave-sms9 krb5kdc[10299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:15.820344 10406 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:53:15.821214 10406 webserver.cc:460] Webserver started at http://127.7.24.193:34179/ using document root <none> and password file <none>
I20260620 14:53:15.821815 10406 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:15.821920 10406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:15.822155 10406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:15.824258 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "b3601be7312645d2b6581c04e883260a"
format_stamp: "Formatted at 2026-06-20 14:53:15 on dist-test-slave-sms9"
server_key: "84724a15439fe0195a6d94f4bde8f255"
server_key_iv: "cfe91eb8f28ead84a5d0cbb79d61ec46"
server_key_version: "encryptionkey@0"
I20260620 14:53:15.824841 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "b3601be7312645d2b6581c04e883260a"
format_stamp: "Formatted at 2026-06-20 14:53:15 on dist-test-slave-sms9"
server_key: "84724a15439fe0195a6d94f4bde8f255"
server_key_iv: "cfe91eb8f28ead84a5d0cbb79d61ec46"
server_key_version: "encryptionkey@0"
I20260620 14:53:15.828943 10406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:53:15.831588 10421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:15.833077 10406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:15.833250 10406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "b3601be7312645d2b6581c04e883260a"
format_stamp: "Formatted at 2026-06-20 14:53:15 on dist-test-slave-sms9"
server_key: "84724a15439fe0195a6d94f4bde8f255"
server_key_iv: "cfe91eb8f28ead84a5d0cbb79d61ec46"
server_key_version: "encryptionkey@0"
I20260620 14:53:15.833376 10406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:15.861411 10406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:15.864593 10406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:15.864830 10406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:15.865435 10406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:15.866324 10406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:15.866405 10406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:15.866473 10406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:15.866571 10406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:15.876703 10406 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:38855
I20260620 14:53:15.876773 10534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:38855 every 8 connection(s)
I20260620 14:53:15.877710 10406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:15.887524 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 10406
I20260620 14:53:15.887744 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:15.888103 7267 external_mini_cluster.cc:1428] Setting key ae58603f69b5ca337047bede97c2d87f
Jun 20 14:53:15 dist-test-slave-sms9 krb5kdc[10299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:15.893637 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:15.880052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:35669 (local address 127.7.24.254:34285)
0620 14:53:15.880209 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:15.880212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:15.881325 (+ 1113us) server_negotiation.cc:407] Connection header received
0620 14:53:15.882719 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:15.882722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:15.882792 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:15.882901 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:15.884165 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:15.884791 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:15.885930 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:15.886185 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:15.886271 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:15.889851 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:15.889858 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:15.889861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:15.889895 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:15.891494 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:15.892052 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:15.892058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:15.892060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:15.892141 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:15.892494 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:15.892499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:15.892501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:15.892714 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:15.892822 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:15.893328 (+ 506us) server_negotiation.cc:299] Negotiation successful
0620 14:53:15.893467 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 14:53:15.894544 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:15.880528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:34285 (local address 127.7.24.193:35669)
0620 14:53:15.881118 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:15.881169 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:15.882446 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:15.883090 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:15.883097 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:15.883464 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:15.884002 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:15.884015 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:15.884920 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:15.884922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:15.885819 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:15.885827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:15.886326 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:15.886328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:15.886547 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:15.887436 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:15.887453 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:15.889630 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:15.891626 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:15.891633 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:15.891651 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:15.891941 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:15.892243 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:15.892249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:15.892251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:15.892397 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:15.892819 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:15.892826 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:15.893083 (+ 257us) client_negotiation.cc:769] Sending connection context
0620 14:53:15.893291 (+ 208us) client_negotiation.cc:240] Negotiation successful
0620 14:53:15.893627 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":160,"threads_started":1}
I20260620 14:53:15.896265 10535 heartbeater.cc:347] Connected to a master server at 127.7.24.254:34285
I20260620 14:53:15.896548 10535 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:15.897087 10535 heartbeater.cc:511] Master 127.7.24.254:34285 requested a full tablet report, sending...
I20260620 14:53:15.899184 10344 ts_manager.cc:194] Registered new tserver with Master: b3601be7312645d2b6581c04e883260a (127.7.24.193:38855)
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:53:15.900777 10344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:35669
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:53:15.919606 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34285
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:16.035486 10540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:16.035797 10540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:16.035878 10540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:16.040683 10540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:16.040758 10540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:16.040859 10540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:16.046368 10540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:16.047595 10540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:16.048676 10540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:16.058843 10545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:16.059023 10540 server_base.cc:1034] running on GCE node
W20260620 14:53:16.058843 10548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:16.059248 10546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:16.059681 10540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:16.060350 10540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:16.061527 10540 hybrid_clock.cc:648] HybridClock initialized: now 1781967196061513 us; error 25 us; skew 500 ppm
Jun 20 14:53:16 dist-test-slave-sms9 krb5kdc[10299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967196, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:16.064536 10540 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:53:16.065693 10540 webserver.cc:460] Webserver started at http://127.7.24.194:46515/ using document root <none> and password file <none>
I20260620 14:53:16.066289 10540 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:16.066356 10540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:16.066612 10540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:16.068755 10540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "450fa490a9d74c66b91c0d3d21a58f02"
format_stamp: "Formatted at 2026-06-20 14:53:16 on dist-test-slave-sms9"
server_key: "b20d15f4e3bb3da0501d4a5417d05c8f"
server_key_iv: "ebbbb4e81f946657e9e48e93445d28e5"
server_key_version: "encryptionkey@0"
I20260620 14:53:16.069315 10540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "450fa490a9d74c66b91c0d3d21a58f02"
format_stamp: "Formatted at 2026-06-20 14:53:16 on dist-test-slave-sms9"
server_key: "b20d15f4e3bb3da0501d4a5417d05c8f"
server_key_iv: "ebbbb4e81f946657e9e48e93445d28e5"
server_key_version: "encryptionkey@0"
I20260620 14:53:16.073189 10540 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:53:16.075798 10555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:16.077171 10540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:16.077319 10540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "450fa490a9d74c66b91c0d3d21a58f02"
format_stamp: "Formatted at 2026-06-20 14:53:16 on dist-test-slave-sms9"
server_key: "b20d15f4e3bb3da0501d4a5417d05c8f"
server_key_iv: "ebbbb4e81f946657e9e48e93445d28e5"
server_key_version: "encryptionkey@0"
I20260620 14:53:16.077422 10540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:16.103483 10540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:16.107043 10540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:16.107204 10540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:16.107744 10540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:16.108536 10540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:16.108584 10540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:16.108624 10540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:16.108640 10540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:16.117681 10540 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:34971
I20260620 14:53:16.117700 10668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:34971 every 8 connection(s)
I20260620 14:53:16.118652 10540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:16.127465 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 10540
I20260620 14:53:16.127593 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:16.128003 7267 external_mini_cluster.cc:1428] Setting key 98273fdec991178a7a37607e3dfa76a5
Jun 20 14:53:16 dist-test-slave-sms9 krb5kdc[10299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967196, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:16.133285 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:16.120499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:48049 (local address 127.7.24.254:34285)
0620 14:53:16.120689 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:16.120693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:16.121556 (+ 863us) server_negotiation.cc:407] Connection header received
0620 14:53:16.123015 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:16.123018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:16.123067 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:16.123145 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:16.124303 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:16.124852 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:16.126004 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:16.126212 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:16.126297 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:16.129311 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:16.129318 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:16.129321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:16.129352 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:16.131152 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:16.131681 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:16.131685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:16.131686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:16.131778 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:16.132134 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:16.132139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:16.132141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:16.132330 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:16.132438 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:16.132961 (+ 523us) server_negotiation.cc:299] Negotiation successful
0620 14:53:16.133111 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 14:53:16.134164 10671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:16.120890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:34285 (local address 127.7.24.194:48049)
0620 14:53:16.121354 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:16.121430 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:16.122809 (+ 1379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:16.123305 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:16.123312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:16.123671 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:16.124156 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:16.124169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:16.124980 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:16.124983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:16.125898 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:16.125907 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:16.126365 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:16.126368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:16.126599 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:16.127246 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:16.127272 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:16.129101 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:16.131272 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:16.131277 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:16.131290 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:16.131550 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:16.131881 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:16.131884 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:16.131886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:16.132034 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:16.132446 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:16.132456 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:16.132713 (+ 257us) client_negotiation.cc:769] Sending connection context
0620 14:53:16.132919 (+ 206us) client_negotiation.cc:240] Negotiation successful
0620 14:53:16.133252 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":106,"threads_started":1}
I20260620 14:53:16.135493 10669 heartbeater.cc:347] Connected to a master server at 127.7.24.254:34285
I20260620 14:53:16.135821 10669 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:16.136317 10669 heartbeater.cc:511] Master 127.7.24.254:34285 requested a full tablet report, sending...
I20260620 14:53:16.137648 10344 ts_manager.cc:194] Registered new tserver with Master: 450fa490a9d74c66b91c0d3d21a58f02 (127.7.24.194:34971)
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:53:16.138370 10344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:48049
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:53:16.157836 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34285
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:16.270599 10674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:16.270888 10674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:16.270933 10674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:16.275027 10674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:16.275089 10674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:16.275148 10674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:16.281164 10674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:16.282817 10674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:16.283915 10674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:16.293883 10682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:16.293888 10679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:16.293888 10680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:16.294229 10674 server_base.cc:1034] running on GCE node
I20260620 14:53:16.294708 10674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:16.295362 10674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:16.296528 10674 hybrid_clock.cc:648] HybridClock initialized: now 1781967196296518 us; error 34 us; skew 500 ppm
Jun 20 14:53:16 dist-test-slave-sms9 krb5kdc[10299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967196, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:16.299435 10674 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:53:16.300407 10674 webserver.cc:460] Webserver started at http://127.7.24.195:39115/ using document root <none> and password file <none>
I20260620 14:53:16.301023 10674 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:16.301117 10674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:16.301333 10674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:16.303527 10674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "806048be5adb4eb68821209c83acd450"
format_stamp: "Formatted at 2026-06-20 14:53:16 on dist-test-slave-sms9"
server_key: "67b777693bdda1ea87dff8393757b508"
server_key_iv: "50bd81202d1b1a0cdf97025403ec92b9"
server_key_version: "encryptionkey@0"
I20260620 14:53:16.304177 10674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "806048be5adb4eb68821209c83acd450"
format_stamp: "Formatted at 2026-06-20 14:53:16 on dist-test-slave-sms9"
server_key: "67b777693bdda1ea87dff8393757b508"
server_key_iv: "50bd81202d1b1a0cdf97025403ec92b9"
server_key_version: "encryptionkey@0"
I20260620 14:53:16.307946 10674 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:53:16.310413 10689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:16.311749 10674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:16.311900 10674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "806048be5adb4eb68821209c83acd450"
format_stamp: "Formatted at 2026-06-20 14:53:16 on dist-test-slave-sms9"
server_key: "67b777693bdda1ea87dff8393757b508"
server_key_iv: "50bd81202d1b1a0cdf97025403ec92b9"
server_key_version: "encryptionkey@0"
I20260620 14:53:16.312018 10674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:16.326350 10674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:16.329402 10674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:16.329612 10674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:16.330209 10674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:16.331064 10674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:16.331133 10674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:16.331204 10674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:16.331257 10674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:16.340592 10674 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:37279
I20260620 14:53:16.340615 10802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:37279 every 8 connection(s)
I20260620 14:53:16.341464 10674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:16.344467 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 10674
I20260620 14:53:16.344599 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:16.344887 7267 external_mini_cluster.cc:1428] Setting key 4d9d5d4311f78bc0adf5d2131d7d9f22
Jun 20 14:53:16 dist-test-slave-sms9 krb5kdc[10299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967196, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:16.355538 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:16.343352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:45785 (local address 127.7.24.254:34285)
0620 14:53:16.343487 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:16.343491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:16.344298 (+ 807us) server_negotiation.cc:407] Connection header received
0620 14:53:16.345979 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:16.345982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:16.346038 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:16.346156 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:16.347512 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:16.348043 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:16.349170 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:16.349392 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:16.349502 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:16.352021 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:16.352025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:16.352034 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:16.352056 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:16.353362 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:16.353914 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:16.353918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:16.353920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:16.354001 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:16.354360 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:16.354363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:16.354365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:16.354605 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:16.354719 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:16.355233 (+ 514us) server_negotiation.cc:299] Negotiation successful
0620 14:53:16.355355 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:53:16.356353 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:16.343668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:34285 (local address 127.7.24.195:45785)
0620 14:53:16.344139 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:16.344200 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:16.345733 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:16.346334 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:16.346343 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:16.346867 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:16.347373 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:16.347386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:16.348170 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:16.348173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:16.349038 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:16.349047 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:16.349541 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:16.349546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:16.349720 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:16.350351 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:16.350368 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:16.351864 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:16.353505 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:16.353511 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:16.353526 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:16.353804 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:16.354095 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:16.354099 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:16.354101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:16.354252 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:16.354727 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:16.354734 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:16.354967 (+ 233us) client_negotiation.cc:769] Sending connection context
0620 14:53:16.355178 (+ 211us) client_negotiation.cc:240] Negotiation successful
0620 14:53:16.355521 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":117,"threads_started":1}
I20260620 14:53:16.357487 10803 heartbeater.cc:347] Connected to a master server at 127.7.24.254:34285
I20260620 14:53:16.357754 10803 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:16.358253 10803 heartbeater.cc:511] Master 127.7.24.254:34285 requested a full tablet report, sending...
I20260620 14:53:16.359313 10344 ts_manager.cc:194] Registered new tserver with Master: 806048be5adb4eb68821209c83acd450 (127.7.24.195:37279)
I20260620 14:53:16.359886 10344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:45785
I20260620 14:53:16.370572 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 14:53:16 dist-test-slave-sms9 krb5kdc[10299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967196, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 20 14:53:16 dist-test-slave-sms9 krb5kdc[10299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967196, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:16.396950 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:16.387839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59302 (local address 127.7.24.254:34285)
0620 14:53:16.387999 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:16.388003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:16.388252 (+ 249us) server_negotiation.cc:407] Connection header received
0620 14:53:16.388352 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:16.388355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:16.388401 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:16.388469 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:16.389374 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:16.389986 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:16.390985 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:16.391231 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:16.391322 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:16.393960 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:16.393964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:16.393965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:16.393988 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:16.395443 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:16.395835 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:16.395838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:16.395839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:16.395912 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:16.396132 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:16.396134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:16.396136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:16.396345 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:16.396449 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:16.396635 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 14:53:16.396728 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260620 14:53:16.397380 10344 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59302
I20260620 14:53:16.398226 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 10406
I20260620 14:53:16.403896 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 10540
I20260620 14:53:16.409456 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 10674
I20260620 14:53:16.415103 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 10312
2026-06-20T14:53:16Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2250 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:16 dist-test-slave-sms9 krb5kdc[10815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:16 dist-test-slave-sms9 krb5kdc[10815](info): set up 2 sockets
Jun 20 14:53:16 dist-test-slave-sms9 krb5kdc[10815](info): commencing operation
krb5kdc: starting...
W20260620 14:53:17.458544 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:17 dist-test-slave-sms9 krb5kdc[10815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:17.585912 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:43129
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:43129
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:17.695163 10828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:17.695459 10828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:17.695502 10828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:17.699625 10828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:17.699699 10828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:17.699718 10828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:17.699734 10828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:17.699750 10828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:17.705798 10828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:43129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:43129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:17.707090 10828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:17.708119 10828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:17.716722 10836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:17.716723 10833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:17.716813 10834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:17.717118 10828 server_base.cc:1034] running on GCE node
I20260620 14:53:17.717631 10828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:17.718801 10828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:17.720006 10828 hybrid_clock.cc:648] HybridClock initialized: now 1781967197719969 us; error 49 us; skew 500 ppm
Jun 20 14:53:17 dist-test-slave-sms9 krb5kdc[10815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967197, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:17.722937 10828 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:17.723896 10828 webserver.cc:460] Webserver started at http://127.7.24.254:42701/ using document root <none> and password file <none>
I20260620 14:53:17.724431 10828 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:17.724489 10828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:17.724665 10828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:17.726904 10828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "1f1e346e5744405980960f191e0dc169"
format_stamp: "Formatted at 2026-06-20 14:53:17 on dist-test-slave-sms9"
server_key: "04cafb662a6faa30cec42cfadb98039f"
server_key_iv: "67caeaa2761cd305d1e3c5fdf90792cb"
server_key_version: "encryptionkey@0"
I20260620 14:53:17.727447 10828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "1f1e346e5744405980960f191e0dc169"
format_stamp: "Formatted at 2026-06-20 14:53:17 on dist-test-slave-sms9"
server_key: "04cafb662a6faa30cec42cfadb98039f"
server_key_iv: "67caeaa2761cd305d1e3c5fdf90792cb"
server_key_version: "encryptionkey@0"
I20260620 14:53:17.731384 10828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 14:53:17.734037 10843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:17.735656 10828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 14:53:17.735770 10828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "1f1e346e5744405980960f191e0dc169"
format_stamp: "Formatted at 2026-06-20 14:53:17 on dist-test-slave-sms9"
server_key: "04cafb662a6faa30cec42cfadb98039f"
server_key_iv: "67caeaa2761cd305d1e3c5fdf90792cb"
server_key_version: "encryptionkey@0"
I20260620 14:53:17.735919 10828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:17.767052 10828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:17.770972 10828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:17.771214 10828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:17.779397 10828 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:43129
I20260620 14:53:17.779420 10895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:43129 every 8 connection(s)
I20260620 14:53:17.780460 10828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:17.782774 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 10828
I20260620 14:53:17.782905 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:17.783183 7267 external_mini_cluster.cc:1428] Setting key 2ee0d14c0045801ae4ee06d0f1b229b5
I20260620 14:53:17.783524 10896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:17.790267 10896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169: Bootstrap starting.
Jun 20 14:53:17 dist-test-slave-sms9 krb5kdc[10815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:17.792760 10896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:17.793675 10896 log.cc:824] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:17.796206 10896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169: No bootstrap required, opened a new log
I20260620 14:53:17.799036 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:17.784731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36130 (local address 127.7.24.254:43129)
0620 14:53:17.785285 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:17.785302 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:17.785340 (+ 38us) server_negotiation.cc:407] Connection header received
0620 14:53:17.786309 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:17.786328 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:17.786819 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:17.787167 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:17.788079 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:17.789063 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:17.789920 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:17.790447 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:17.790665 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:17.793228 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:17.793235 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:17.793253 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:17.793285 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:17.795520 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:17.796020 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:17.796026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:17.796032 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:17.796176 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:17.796567 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:17.796570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:17.796572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:17.797127 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:17.797318 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:17.797621 (+ 303us) server_negotiation.cc:299] Negotiation successful
0620 14:53:17.797923 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":141,"threads_started":1}
I20260620 14:53:17.800909 10896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1e346e5744405980960f191e0dc169" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43129 } }
I20260620 14:53:17.801147 10896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:17.801213 10896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1e346e5744405980960f191e0dc169, State: Initialized, Role: FOLLOWER
I20260620 14:53:17.801697 10896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1e346e5744405980960f191e0dc169" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43129 } }
I20260620 14:53:17.801839 10896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:17.801924 10896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:17.802017 10896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:17.803187 10896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1e346e5744405980960f191e0dc169" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43129 } }
I20260620 14:53:17.803551 10896 leader_election.cc:302] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f1e346e5744405980960f191e0dc169; no voters:
I20260620 14:53:17.803879 10896 leader_election.cc:288] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:17.804101 10901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:17.804389 10901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [term 1 LEADER]: Becoming Leader. State: Replica: 1f1e346e5744405980960f191e0dc169, State: Running, Role: LEADER
I20260620 14:53:17.805002 10901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1e346e5744405980960f191e0dc169" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43129 } }
I20260620 14:53:17.805251 10896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:17.808228 10903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f1e346e5744405980960f191e0dc169. Latest consensus state: current_term: 1 leader_uuid: "1f1e346e5744405980960f191e0dc169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1e346e5744405980960f191e0dc169" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43129 } } }
I20260620 14:53:17.808679 10903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:17.808420 10902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f1e346e5744405980960f191e0dc169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1e346e5744405980960f191e0dc169" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43129 } } }
I20260620 14:53:17.808991 10902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169 [sys.catalog]: This master's current role is: LEADER
W20260620 14:53:17.810953 10915 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:53:17.811048 10915 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:53:17.811149 10916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:17.811865 10916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:17.818053 10916 catalog_manager.cc:1347] Generated new cluster ID: 0d197971fb9e4cbda68cba7676aa1f7b
I20260620 14:53:17.818152 10916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:17.851291 10916 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:17.852352 10916 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:17.859643 10916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f1e346e5744405980960f191e0dc169: Generated new TSK 0
I20260620 14:53:17.860440 10916 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:53:17.898319 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43129
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:18.049957 10922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:18.050318 10922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:18.050381 10922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:18.054658 10922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:18.054728 10922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:18.054793 10922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:18.060104 10922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:18.061127 10922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:18.062160 10922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:18.071767 10928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:18.071784 10922 server_base.cc:1034] running on GCE node
W20260620 14:53:18.071692 10930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:18.071751 10927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:18.072427 10922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:18.073043 10922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:18.074218 10922 hybrid_clock.cc:648] HybridClock initialized: now 1781967198074204 us; error 25 us; skew 500 ppm
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[10815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967198, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:18.076892 10922 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:53:18.077720 10922 webserver.cc:460] Webserver started at http://127.7.24.193:37881/ using document root <none> and password file <none>
I20260620 14:53:18.078279 10922 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:18.078368 10922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:18.078636 10922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:18.080631 10922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "7cf677b0049a46f5ae3e77212907f60e"
format_stamp: "Formatted at 2026-06-20 14:53:18 on dist-test-slave-sms9"
server_key: "cc10434c763bf6faf24eb6e2e2988a0c"
server_key_iv: "1a3fb5d4448df1719761500e89b3709e"
server_key_version: "encryptionkey@0"
I20260620 14:53:18.081198 10922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "7cf677b0049a46f5ae3e77212907f60e"
format_stamp: "Formatted at 2026-06-20 14:53:18 on dist-test-slave-sms9"
server_key: "cc10434c763bf6faf24eb6e2e2988a0c"
server_key_iv: "1a3fb5d4448df1719761500e89b3709e"
server_key_version: "encryptionkey@0"
I20260620 14:53:18.084793 10922 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 14:53:18.087136 10937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:18.088260 10922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:18.088403 10922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "7cf677b0049a46f5ae3e77212907f60e"
format_stamp: "Formatted at 2026-06-20 14:53:18 on dist-test-slave-sms9"
server_key: "cc10434c763bf6faf24eb6e2e2988a0c"
server_key_iv: "1a3fb5d4448df1719761500e89b3709e"
server_key_version: "encryptionkey@0"
I20260620 14:53:18.088512 10922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:18.098191 10922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:18.100831 10922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:18.101029 10922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:18.101576 10922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:18.102348 10922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:18.102417 10922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:18.102510 10922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:18.102566 10922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:18.111475 10922 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:42403
I20260620 14:53:18.111505 11050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:42403 every 8 connection(s)
I20260620 14:53:18.112308 10922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:18.115734 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 10922
I20260620 14:53:18.115866 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:18.116240 7267 external_mini_cluster.cc:1428] Setting key e63a69665c11dcd0d8649cc8c8b2a026
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[10815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967198, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:53:18.127005 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:18.114231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:45541 (local address 127.7.24.254:43129)
0620 14:53:18.114392 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:18.114395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:18.115288 (+ 893us) server_negotiation.cc:407] Connection header received
0620 14:53:18.117131 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:18.117134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:18.117184 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:18.117314 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:18.118583 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.119286 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.120461 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.120695 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.120819 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:18.123314 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:18.123318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:18.123320 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:18.123342 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:18.124838 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:18.125380 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:18.125383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:18.125385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:18.125464 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:18.125816 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:18.125820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:18.125821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:18.126005 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:18.126117 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:18.126695 (+ 578us) server_negotiation.cc:299] Negotiation successful
0620 14:53:18.126829 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
I20260620 14:53:18.127827 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:18.114671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:43129 (local address 127.7.24.193:45541)
0620 14:53:18.115131 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:18.115178 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:18.116894 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:18.117526 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:18.117535 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:18.117953 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:18.118414 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:18.118427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.119448 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.119451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:18.120329 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:18.120338 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.120848 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.120853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:18.121035 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:18.121662 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:18.121678 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:18.123145 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:18.124974 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:18.124980 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:18.124994 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:18.125260 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:18.125565 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:18.125568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:18.125570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:18.125706 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:18.126114 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:18.126122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:18.126395 (+ 273us) client_negotiation.cc:769] Sending connection context
0620 14:53:18.126637 (+ 242us) client_negotiation.cc:240] Negotiation successful
0620 14:53:18.126956 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":98,"threads_started":1}
I20260620 14:53:18.129086 11051 heartbeater.cc:347] Connected to a master server at 127.7.24.254:43129
I20260620 14:53:18.129370 11051 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:18.129956 11051 heartbeater.cc:511] Master 127.7.24.254:43129 requested a full tablet report, sending...
I20260620 14:53:18.131757 10860 ts_manager.cc:194] Registered new tserver with Master: 7cf677b0049a46f5ae3e77212907f60e (127.7.24.193:42403)
I20260620 14:53:18.132931 10860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:45541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:53:18.143477 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43129
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:18.256220 11056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:18.256544 11056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:18.256621 11056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:18.260900 11056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:18.261000 11056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:18.261096 11056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:18.266396 11056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:18.267573 11056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:18.268605 11056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:18.278343 11062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:18.278343 11064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:18.278479 11056 server_base.cc:1034] running on GCE node
W20260620 14:53:18.278375 11061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:18.279091 11056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:18.279780 11056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:18.280946 11056 hybrid_clock.cc:648] HybridClock initialized: now 1781967198280911 us; error 46 us; skew 500 ppm
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[10815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967198, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:18.283833 11056 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:53:18.284684 11056 webserver.cc:460] Webserver started at http://127.7.24.194:41047/ using document root <none> and password file <none>
I20260620 14:53:18.285219 11056 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:18.285277 11056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:18.285447 11056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:18.287701 11056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "bcb12b8946674b08951ae12bd762d977"
format_stamp: "Formatted at 2026-06-20 14:53:18 on dist-test-slave-sms9"
server_key: "7732924c968510567f4bd9e7666b7fa3"
server_key_iv: "2890e99b77af98e8dfcdc991db555f05"
server_key_version: "encryptionkey@0"
I20260620 14:53:18.288410 11056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "bcb12b8946674b08951ae12bd762d977"
format_stamp: "Formatted at 2026-06-20 14:53:18 on dist-test-slave-sms9"
server_key: "7732924c968510567f4bd9e7666b7fa3"
server_key_iv: "2890e99b77af98e8dfcdc991db555f05"
server_key_version: "encryptionkey@0"
I20260620 14:53:18.293013 11056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:53:18.295552 11071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:18.296854 11056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:18.296955 11056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "bcb12b8946674b08951ae12bd762d977"
format_stamp: "Formatted at 2026-06-20 14:53:18 on dist-test-slave-sms9"
server_key: "7732924c968510567f4bd9e7666b7fa3"
server_key_iv: "2890e99b77af98e8dfcdc991db555f05"
server_key_version: "encryptionkey@0"
I20260620 14:53:18.297055 11056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:18.325974 11056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:18.329355 11056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:18.329512 11056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:18.330045 11056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:18.330876 11056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:18.330924 11056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:18.330964 11056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:18.330981 11056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:18.340407 11056 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:39617
I20260620 14:53:18.340437 11184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:39617 every 8 connection(s)
I20260620 14:53:18.341310 11056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:18.350335 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 11056
I20260620 14:53:18.350430 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:18.350750 7267 external_mini_cluster.cc:1428] Setting key 5d18b866bcaf3a7c5561f3cd4c415589
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[10815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967198, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:18.355489 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:18.343281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:42617 (local address 127.7.24.254:43129)
0620 14:53:18.343454 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:18.343459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:18.344263 (+ 804us) server_negotiation.cc:407] Connection header received
0620 14:53:18.345687 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:18.345690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:18.345736 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:18.345811 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:18.347117 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.347637 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.348838 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.349052 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.349137 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:18.351835 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:18.351843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:18.351845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:18.351869 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:18.353359 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:18.353890 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:18.353894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:18.353896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:18.353977 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:18.354321 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:18.354326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:18.354327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:18.354582 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:18.354690 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:18.355195 (+ 505us) server_negotiation.cc:299] Negotiation successful
0620 14:53:18.355323 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 14:53:18.356295 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:18.343627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:43129 (local address 127.7.24.194:42617)
0620 14:53:18.344083 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:18.344146 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:18.345483 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:18.345982 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:18.345988 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:18.346373 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:18.346971 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:18.346985 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.347760 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.347763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:18.348698 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:18.348708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.349216 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.349220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:18.349366 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:18.349983 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:18.349999 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:18.351663 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:18.353483 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:18.353489 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:18.353505 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:18.353773 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:18.354073 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:18.354077 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:18.354079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:18.354221 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:18.354698 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:18.354706 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:18.354957 (+ 251us) client_negotiation.cc:769] Sending connection context
0620 14:53:18.355151 (+ 194us) client_negotiation.cc:240] Negotiation successful
0620 14:53:18.355467 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":101,"threads_started":1}
I20260620 14:53:18.357621 11185 heartbeater.cc:347] Connected to a master server at 127.7.24.254:43129
I20260620 14:53:18.357931 11185 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:18.358429 11185 heartbeater.cc:511] Master 127.7.24.254:43129 requested a full tablet report, sending...
I20260620 14:53:18.359624 10860 ts_manager.cc:194] Registered new tserver with Master: bcb12b8946674b08951ae12bd762d977 (127.7.24.194:39617)
I20260620 14:53:18.360474 10860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:42617
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:53:18.381276 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43129
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:18.492964 11190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:18.493232 11190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:18.493283 11190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:18.498219 11190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:18.498317 11190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:18.498610 11190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:18.508937 11190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:18.510399 11190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:18.512204 11190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:18.524457 11195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:18.524470 11196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:18.524457 11198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:18.524863 11190 server_base.cc:1034] running on GCE node
I20260620 14:53:18.525213 11190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:18.525830 11190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:18.526980 11190 hybrid_clock.cc:648] HybridClock initialized: now 1781967198526950 us; error 43 us; skew 500 ppm
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[10815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967198, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:18.529752 11190 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:53:18.530620 11190 webserver.cc:460] Webserver started at http://127.7.24.195:33183/ using document root <none> and password file <none>
I20260620 14:53:18.531124 11190 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:18.531180 11190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:18.531353 11190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:18.533311 11190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "05950224fb184ebea7e8cdfac5ee827c"
format_stamp: "Formatted at 2026-06-20 14:53:18 on dist-test-slave-sms9"
server_key: "e2ac0b9dbb9b5a5b3d4fbbcfe2a7d9af"
server_key_iv: "92144b48388cc29570824f6995f71f4d"
server_key_version: "encryptionkey@0"
I20260620 14:53:18.533809 11190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "05950224fb184ebea7e8cdfac5ee827c"
format_stamp: "Formatted at 2026-06-20 14:53:18 on dist-test-slave-sms9"
server_key: "e2ac0b9dbb9b5a5b3d4fbbcfe2a7d9af"
server_key_iv: "92144b48388cc29570824f6995f71f4d"
server_key_version: "encryptionkey@0"
I20260620 14:53:18.537446 11190 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260620 14:53:18.539899 11205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:18.541162 11190 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:18.541280 11190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "05950224fb184ebea7e8cdfac5ee827c"
format_stamp: "Formatted at 2026-06-20 14:53:18 on dist-test-slave-sms9"
server_key: "e2ac0b9dbb9b5a5b3d4fbbcfe2a7d9af"
server_key_iv: "92144b48388cc29570824f6995f71f4d"
server_key_version: "encryptionkey@0"
I20260620 14:53:18.541391 11190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:18.562811 11190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:18.565876 11190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:18.566085 11190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:18.566776 11190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:18.567608 11190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:18.567682 11190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:18.567759 11190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:18.567808 11190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:18.576756 11190 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:43257
I20260620 14:53:18.576772 11318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:43257 every 8 connection(s)
I20260620 14:53:18.577612 11190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:18.577893 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 11190
I20260620 14:53:18.578024 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:18.578353 7267 external_mini_cluster.cc:1428] Setting key c88621b791b17071176591e5c88df385
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[10815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967198, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:18.591745 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:18.579612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:43615 (local address 127.7.24.254:43129)
0620 14:53:18.579790 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:18.579794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:18.580886 (+ 1092us) server_negotiation.cc:407] Connection header received
0620 14:53:18.582190 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:18.582193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:18.582238 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:18.582308 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:18.583514 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.584053 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.585155 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.585381 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.585476 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:18.588180 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:18.588182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:18.588185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:18.588208 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:18.589549 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:18.590023 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:18.590026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:18.590027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:18.590098 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:18.590410 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:18.590412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:18.590413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:18.590667 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:18.590753 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:18.591429 (+ 676us) server_negotiation.cc:299] Negotiation successful
0620 14:53:18.591561 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260620 14:53:18.592760 11321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:18.580119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:43129 (local address 127.7.24.195:43615)
0620 14:53:18.580715 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:18.580778 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:18.582003 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:18.582520 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:18.582528 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:18.582918 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:18.583376 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:18.583387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.584178 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.584181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:18.585025 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:18.585033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.585515 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.585518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:18.585661 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:18.586294 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:18.586310 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:18.588006 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:18.589673 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:18.589678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:18.589692 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:18.589936 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:18.590194 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:18.590196 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:18.590197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:18.590321 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:18.590797 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:18.590802 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:18.591136 (+ 334us) client_negotiation.cc:769] Sending connection context
0620 14:53:18.591375 (+ 239us) client_negotiation.cc:240] Negotiation successful
0620 14:53:18.591727 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":132,"threads_started":1}
I20260620 14:53:18.593933 11319 heartbeater.cc:347] Connected to a master server at 127.7.24.254:43129
I20260620 14:53:18.594197 11319 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:18.594748 11319 heartbeater.cc:511] Master 127.7.24.254:43129 requested a full tablet report, sending...
I20260620 14:53:18.595805 10860 ts_manager.cc:194] Registered new tserver with Master: 05950224fb184ebea7e8cdfac5ee827c (127.7.24.195:43257)
I20260620 14:53:18.596343 10860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:43615
I20260620 14:53:18.604164 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[10815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[10815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:53:18.630019 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:18.620690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.7.24.193:42403)
0620 14:53:18.621003 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:18.621017 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:18.621039 (+ 22us) server_negotiation.cc:407] Connection header received
0620 14:53:18.621088 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:18.621093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:18.621276 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:18.621395 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:18.622042 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.622899 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.623704 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.624073 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.624154 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:18.626637 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:18.626645 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:18.626658 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:18.626680 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:18.628126 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:18.628672 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:18.628685 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:18.628686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:18.628765 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:18.629052 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:18.629055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:18.629057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:18.629293 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:18.629541 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:18.629704 (+ 163us) server_negotiation.cc:299] Negotiation successful
0620 14:53:18.629867 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":121,"threads_started":1}
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[10815](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:18.641484 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:18.632956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36132 (local address 127.7.24.254:43129)
0620 14:53:18.633117 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:18.633120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:18.633134 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:18.633306 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:18.633309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:18.633356 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:18.633438 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:18.633994 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.634622 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.635392 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:18.635627 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:18.635755 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:18.638134 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:18.638138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:18.638141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:18.638167 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:18.639615 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:18.640116 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:18.640121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:18.640123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:18.640239 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:18.640536 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:18.640541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:18.640543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:18.640778 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:18.640921 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:18.641181 (+ 260us) server_negotiation.cc:299] Negotiation successful
0620 14:53:18.641305 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260620 14:53:18.641894 10860 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36132
I20260620 14:53:18.642560 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 10922
I20260620 14:53:18.648480 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 11056
I20260620 14:53:18.653671 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 11190
I20260620 14:53:18.659196 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 10828
2026-06-20T14:53:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2243 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[11331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[11331](info): set up 2 sockets
Jun 20 14:53:18 dist-test-slave-sms9 krb5kdc[11331](info): commencing operation
krb5kdc: starting...
W20260620 14:53:19.715027 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:19 dist-test-slave-sms9 krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:19.839644 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:35571
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:35571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:19.949343 11344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:19.949589 11344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:19.949630 11344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:19.953636 11344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:19.953692 11344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:19.953711 11344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:19.953727 11344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:19.953742 11344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:19.959043 11344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:35571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:35571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:19.960134 11344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:19.961150 11344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:19.969942 11350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:19.969981 11349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:19.970041 11344 server_base.cc:1034] running on GCE node
W20260620 14:53:19.969982 11352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:19.970902 11344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:19.972093 11344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:19.973271 11344 hybrid_clock.cc:648] HybridClock initialized: now 1781967199973252 us; error 32 us; skew 500 ppm
Jun 20 14:53:19 dist-test-slave-sms9 krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967199, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:19.979949 11344 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:19.980734 11344 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:19.980831 11344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:19.981065 11344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:19.983201 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "347227bf0def4f72ac63a13143fceee2"
format_stamp: "Formatted at 2026-06-20 14:53:19 on dist-test-slave-sms9"
server_key: "2fbb792c5d7207e811b16bcf9d52a2d8"
server_key_iv: "7bfea99f06c2c3842faa65ede03787da"
server_key_version: "encryptionkey@0"
I20260620 14:53:19.983776 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "347227bf0def4f72ac63a13143fceee2"
format_stamp: "Formatted at 2026-06-20 14:53:19 on dist-test-slave-sms9"
server_key: "2fbb792c5d7207e811b16bcf9d52a2d8"
server_key_iv: "7bfea99f06c2c3842faa65ede03787da"
server_key_version: "encryptionkey@0"
I20260620 14:53:19.987417 11344 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:53:19.989738 11358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:19.990983 11344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:19.991130 11344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "347227bf0def4f72ac63a13143fceee2"
format_stamp: "Formatted at 2026-06-20 14:53:19 on dist-test-slave-sms9"
server_key: "2fbb792c5d7207e811b16bcf9d52a2d8"
server_key_iv: "7bfea99f06c2c3842faa65ede03787da"
server_key_version: "encryptionkey@0"
I20260620 14:53:19.991257 11344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:20.011233 11344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:20.014689 11344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:20.014907 11344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:20.021879 11344 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:35571
I20260620 14:53:20.022137 11410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:35571 every 8 connection(s)
I20260620 14:53:20.022570 11344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:20.025195 11411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:20.026347 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 11344
I20260620 14:53:20.026448 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:20.026767 7267 external_mini_cluster.cc:1428] Setting key 0591530677582dc23b9b41e5b77888f2
I20260620 14:53:20.031342 11411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2: Bootstrap starting.
I20260620 14:53:20.034116 11411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:20.035193 11411 log.cc:824] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2: Log is configured to *not* fsync() on all Append() calls
Jun 20 14:53:20 dist-test-slave-sms9 krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:20.037376 11411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2: No bootstrap required, opened a new log
I20260620 14:53:20.041165 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.028141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34008 (local address 127.7.24.254:35571)
0620 14:53:20.028532 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:20.028546 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:20.028578 (+ 32us) server_negotiation.cc:407] Connection header received
0620 14:53:20.029199 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:20.029214 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:20.029530 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:20.029795 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:20.030724 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.031712 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.032847 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.033312 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.033425 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.036643 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:20.036648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:20.036661 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:20.036685 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:20.038551 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:20.038969 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:20.038975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:20.038987 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:20.039088 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:20.039372 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:20.039375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:20.039377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:20.039740 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:20.039931 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:20.040203 (+ 272us) server_negotiation.cc:299] Negotiation successful
0620 14:53:20.040479 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":136,"threads_started":1}
I20260620 14:53:20.041465 11411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347227bf0def4f72ac63a13143fceee2" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35571 } }
I20260620 14:53:20.041712 11411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:20.041810 11411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 347227bf0def4f72ac63a13143fceee2, State: Initialized, Role: FOLLOWER
I20260620 14:53:20.042321 11411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347227bf0def4f72ac63a13143fceee2" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35571 } }
I20260620 14:53:20.042508 11411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:20.042580 11411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:20.042680 11411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:20.043891 11411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347227bf0def4f72ac63a13143fceee2" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35571 } }
I20260620 14:53:20.044200 11411 leader_election.cc:302] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 347227bf0def4f72ac63a13143fceee2; no voters:
I20260620 14:53:20.044461 11411 leader_election.cc:288] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:20.044584 11416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:20.044819 11416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [term 1 LEADER]: Becoming Leader. State: Replica: 347227bf0def4f72ac63a13143fceee2, State: Running, Role: LEADER
I20260620 14:53:20.045125 11416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347227bf0def4f72ac63a13143fceee2" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35571 } }
I20260620 14:53:20.045877 11411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:20.046250 11417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 347227bf0def4f72ac63a13143fceee2. Latest consensus state: current_term: 1 leader_uuid: "347227bf0def4f72ac63a13143fceee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347227bf0def4f72ac63a13143fceee2" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35571 } } }
I20260620 14:53:20.046392 11417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:20.046732 11418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "347227bf0def4f72ac63a13143fceee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347227bf0def4f72ac63a13143fceee2" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35571 } } }
I20260620 14:53:20.046845 11418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2 [sys.catalog]: This master's current role is: LEADER
W20260620 14:53:20.050722 11432 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:53:20.050814 11432 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:53:20.050884 11420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:20.051554 11420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:20.057156 11420 catalog_manager.cc:1347] Generated new cluster ID: ec2e788732fd4868a645584c6f1e5082
I20260620 14:53:20.057247 11420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:20.091223 11420 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:20.092721 11420 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:20.101884 11420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2: Generated new TSK 0
I20260620 14:53:20.102831 11420 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:53:20.129680 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35571
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:20.246032 11437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:20.246377 11437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:20.246460 11437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:20.250938 11437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:20.251053 11437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:20.251155 11437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:20.256829 11437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:20.257997 11437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:20.259065 11437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:20.268517 11443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:20.268525 11437 server_base.cc:1034] running on GCE node
W20260620 14:53:20.268537 11442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:20.268549 11445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:20.269196 11437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:20.269829 11437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:20.271014 11437 hybrid_clock.cc:648] HybridClock initialized: now 1781967200270999 us; error 27 us; skew 500 ppm
Jun 20 14:53:20 dist-test-slave-sms9 krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967200, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:20.273797 11437 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:53:20.274557 11437 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:20.274627 11437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:20.274798 11437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:20.276820 11437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "77f07c543f074011955b74d4c2591d28"
format_stamp: "Formatted at 2026-06-20 14:53:20 on dist-test-slave-sms9"
server_key: "5b3c9b828e9625e6e9aacef398b365d5"
server_key_iv: "a39e6a0c48e469a7c439d42ec01f3687"
server_key_version: "encryptionkey@0"
I20260620 14:53:20.277328 11437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "77f07c543f074011955b74d4c2591d28"
format_stamp: "Formatted at 2026-06-20 14:53:20 on dist-test-slave-sms9"
server_key: "5b3c9b828e9625e6e9aacef398b365d5"
server_key_iv: "a39e6a0c48e469a7c439d42ec01f3687"
server_key_version: "encryptionkey@0"
I20260620 14:53:20.281014 11437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 14:53:20.283684 11451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:20.285030 11437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:20.285128 11437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "77f07c543f074011955b74d4c2591d28"
format_stamp: "Formatted at 2026-06-20 14:53:20 on dist-test-slave-sms9"
server_key: "5b3c9b828e9625e6e9aacef398b365d5"
server_key_iv: "a39e6a0c48e469a7c439d42ec01f3687"
server_key_version: "encryptionkey@0"
I20260620 14:53:20.285219 11437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:20.294350 11437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:20.298169 11437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:20.298368 11437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:20.299126 11437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:20.299962 11437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:20.300007 11437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:20.300045 11437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:20.300061 11437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:20.308779 11437 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:43627
I20260620 14:53:20.308794 11564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:43627 every 8 connection(s)
I20260620 14:53:20.309487 11437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:20.316439 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 11437
I20260620 14:53:20.316597 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:20.316913 7267 external_mini_cluster.cc:1428] Setting key 7116b1a8a4bc0fccc380e4d9b2994fff
Jun 20 14:53:20 dist-test-slave-sms9 krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967200, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:20.324640 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.311314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:50593 (local address 127.7.24.254:35571)
0620 14:53:20.311482 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:20.311486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:20.312355 (+ 869us) server_negotiation.cc:407] Connection header received
0620 14:53:20.313775 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:20.313778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:20.313837 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:20.313917 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:20.315289 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.315850 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.317240 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.317568 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.317700 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.320726 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:20.320729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:20.320732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:20.320768 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:20.322341 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:20.322964 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:20.322969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:20.322971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:20.323062 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:20.323483 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:20.323486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:20.323487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:20.323689 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:20.323797 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:20.324329 (+ 532us) server_negotiation.cc:299] Negotiation successful
0620 14:53:20.324461 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260620 14:53:20.325495 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.311682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:35571 (local address 127.7.24.193:50593)
0620 14:53:20.312193 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:20.312238 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:20.313556 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:20.314086 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:20.314092 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:20.314510 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:20.315043 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.315058 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.315992 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.315998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.317105 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.317114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.317742 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.317746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.317918 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.318579 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:20.318601 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:20.320540 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:20.322501 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:20.322510 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:20.322528 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:20.322861 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:20.323164 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:20.323168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:20.323170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:20.323378 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:20.323797 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:20.323804 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:20.324079 (+ 275us) client_negotiation.cc:769] Sending connection context
0620 14:53:20.324278 (+ 199us) client_negotiation.cc:240] Negotiation successful
0620 14:53:20.324627 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":116,"threads_started":1}
I20260620 14:53:20.327066 11565 heartbeater.cc:347] Connected to a master server at 127.7.24.254:35571
I20260620 14:53:20.327332 11565 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:53:20.327860 11565 heartbeater.cc:511] Master 127.7.24.254:35571 requested a full tablet report, sending...
I20260620 14:53:20.329483 11375 ts_manager.cc:194] Registered new tserver with Master: 77f07c543f074011955b74d4c2591d28 (127.7.24.193:43627)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
I20260620 14:53:20.330873 11375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:50593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:53:20.347258 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35571
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:20.457190 11570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:20.457454 11570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:20.457502 11570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:20.461632 11570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:20.461704 11570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:20.461800 11570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:20.467445 11570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:20.468472 11570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:20.469508 11570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:20.478518 11575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:20.478513 11576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:20.478513 11578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:20.478826 11570 server_base.cc:1034] running on GCE node
I20260620 14:53:20.479272 11570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:20.479880 11570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:20.481057 11570 hybrid_clock.cc:648] HybridClock initialized: now 1781967200481031 us; error 44 us; skew 500 ppm
Jun 20 14:53:20 dist-test-slave-sms9 krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967200, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:20.483811 11570 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:53:20.484562 11570 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:20.484663 11570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:20.484894 11570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:20.487095 11570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "0c8d094cdbd34d4c89ea64a1785bffdc"
format_stamp: "Formatted at 2026-06-20 14:53:20 on dist-test-slave-sms9"
server_key: "6f1dd3941d5f214b605eae3583654642"
server_key_iv: "a9acc843f36573ccf27491145b8dac5e"
server_key_version: "encryptionkey@0"
I20260620 14:53:20.487680 11570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "0c8d094cdbd34d4c89ea64a1785bffdc"
format_stamp: "Formatted at 2026-06-20 14:53:20 on dist-test-slave-sms9"
server_key: "6f1dd3941d5f214b605eae3583654642"
server_key_iv: "a9acc843f36573ccf27491145b8dac5e"
server_key_version: "encryptionkey@0"
I20260620 14:53:20.493378 11570 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260620 14:53:20.497385 11584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:20.499228 11570 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:20.499361 11570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "0c8d094cdbd34d4c89ea64a1785bffdc"
format_stamp: "Formatted at 2026-06-20 14:53:20 on dist-test-slave-sms9"
server_key: "6f1dd3941d5f214b605eae3583654642"
server_key_iv: "a9acc843f36573ccf27491145b8dac5e"
server_key_version: "encryptionkey@0"
I20260620 14:53:20.499665 11570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:20.523489 11570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:20.527628 11570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:20.527833 11570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:20.528482 11570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:20.529326 11570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:20.529399 11570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:20.529467 11570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:20.529502 11570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:20.539999 11570 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:38147
I20260620 14:53:20.540016 11697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:38147 every 8 connection(s)
I20260620 14:53:20.540597 11570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:20.543604 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 11570
I20260620 14:53:20.543824 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:20.544126 7267 external_mini_cluster.cc:1428] Setting key 4537f9be37750b614a74841fa94f6c68
Jun 20 14:53:20 dist-test-slave-sms9 krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967200, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:53:20.555187 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.542253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:47151 (local address 127.7.24.254:35571)
0620 14:53:20.542388 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:20.542392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:20.543614 (+ 1222us) server_negotiation.cc:407] Connection header received
0620 14:53:20.544605 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:20.544608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:20.544730 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:20.544824 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:20.546182 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.546877 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.548336 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.548554 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.548715 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.551335 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:20.551339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:20.551342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:20.551369 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:20.552938 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:20.553525 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:20.553529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:20.553531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:20.553620 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:20.553981 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:20.553985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:20.553987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:20.554185 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:20.554292 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:20.554895 (+ 603us) server_negotiation.cc:299] Negotiation successful
0620 14:53:20.555018 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
I20260620 14:53:20.556173 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.542640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:35571 (local address 127.7.24.194:47151)
0620 14:53:20.543455 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:20.543496 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:20.544397 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:20.545010 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:20.545017 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:20.545371 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:20.545943 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.545958 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.547046 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.547053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.548214 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.548225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.548752 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.548756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.548926 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.549535 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:20.549552 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:20.551162 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:20.553076 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:20.553082 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:20.553098 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:20.553428 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:20.553727 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:20.553730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:20.553732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:20.553876 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:20.554291 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:20.554299 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:20.554647 (+ 348us) client_negotiation.cc:769] Sending connection context
0620 14:53:20.554847 (+ 200us) client_negotiation.cc:240] Negotiation successful
0620 14:53:20.555195 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":123,"threads_started":1}
I20260620 14:53:20.557353 11698 heartbeater.cc:347] Connected to a master server at 127.7.24.254:35571
I20260620 14:53:20.557672 11698 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:20.558140 11698 heartbeater.cc:511] Master 127.7.24.254:35571 requested a full tablet report, sending...
I20260620 14:53:20.559262 11375 ts_manager.cc:194] Registered new tserver with Master: 0c8d094cdbd34d4c89ea64a1785bffdc (127.7.24.194:38147)
I20260620 14:53:20.559957 11375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:47151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:53:20.573132 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35571
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:20.686642 11703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:20.687002 11703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:20.687088 11703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:20.691454 11703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:20.691561 11703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:20.691655 11703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:20.697587 11703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:20.699011 11703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:20.700166 11703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:20.710644 11709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:20.710700 11711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:20.710582 11708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:20.710863 11703 server_base.cc:1034] running on GCE node
I20260620 14:53:20.711273 11703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:20.711935 11703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:20.713142 11703 hybrid_clock.cc:648] HybridClock initialized: now 1781967200713111 us; error 60 us; skew 500 ppm
Jun 20 14:53:20 dist-test-slave-sms9 krb5kdc[11331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967200, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:20.715952 11703 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:53:20.716701 11703 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:20.716791 11703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:20.717011 11703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:20.719111 11703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc"
format_stamp: "Formatted at 2026-06-20 14:53:20 on dist-test-slave-sms9"
server_key: "fe0bc4acc5beffda229a9045e04455ae"
server_key_iv: "81e0ce02d96468d9dd7cda3150ade042"
server_key_version: "encryptionkey@0"
I20260620 14:53:20.719666 11703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc"
format_stamp: "Formatted at 2026-06-20 14:53:20 on dist-test-slave-sms9"
server_key: "fe0bc4acc5beffda229a9045e04455ae"
server_key_iv: "81e0ce02d96468d9dd7cda3150ade042"
server_key_version: "encryptionkey@0"
I20260620 14:53:20.723538 11703 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 14:53:20.726060 11717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:20.727344 11703 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 14:53:20.727486 11703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc"
format_stamp: "Formatted at 2026-06-20 14:53:20 on dist-test-slave-sms9"
server_key: "fe0bc4acc5beffda229a9045e04455ae"
server_key_iv: "81e0ce02d96468d9dd7cda3150ade042"
server_key_version: "encryptionkey@0"
I20260620 14:53:20.727602 11703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:20.742935 11703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:20.745784 11703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:20.745970 11703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:20.746598 11703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:20.747460 11703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:20.747527 11703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:20.747604 11703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:20.747649 11703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:20.756421 11703 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:41871
I20260620 14:53:20.756436 11830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:41871 every 8 connection(s)
I20260620 14:53:20.757006 11703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:20.761015 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 11703
I20260620 14:53:20.761152 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:20.761430 7267 external_mini_cluster.cc:1428] Setting key d421ee86ef94d5f008b0ba6fca6e7f84
Jun 20 14:53:20 dist-test-slave-sms9 krb5kdc[11331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967200, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:20.770057 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.758634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:33217 (local address 127.7.24.254:35571)
0620 14:53:20.758797 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:20.758802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:20.759989 (+ 1187us) server_negotiation.cc:407] Connection header received
0620 14:53:20.760820 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:20.760825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:20.760889 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:20.761001 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:20.762226 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.762829 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.763928 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.764149 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.764257 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.766620 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:20.766623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:20.766626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:20.766655 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:20.768109 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:20.768611 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:20.768615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:20.768617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:20.768700 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:20.769021 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:20.769026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:20.769027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:20.769202 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:20.769306 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:20.769774 (+ 468us) server_negotiation.cc:299] Negotiation successful
0620 14:53:20.769891 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 14:53:20.770884 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.759085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:35571 (local address 127.7.24.195:33217)
0620 14:53:20.759840 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:20.759879 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:20.760582 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:20.761170 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:20.761176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:20.761618 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:20.762081 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.762094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.762947 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.762951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.763809 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.763817 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.764296 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.764300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.764467 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.765047 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:20.765063 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:20.766470 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:20.768237 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:20.768249 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:20.768264 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:20.768512 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:20.768795 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:20.768800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:20.768802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:20.768932 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:20.769304 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:20.769310 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:20.769543 (+ 233us) client_negotiation.cc:769] Sending connection context
0620 14:53:20.769723 (+ 180us) client_negotiation.cc:240] Negotiation successful
0620 14:53:20.770023 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":116,"threads_started":1}
I20260620 14:53:20.771932 11831 heartbeater.cc:347] Connected to a master server at 127.7.24.254:35571
I20260620 14:53:20.772151 11831 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:20.772540 11831 heartbeater.cc:511] Master 127.7.24.254:35571 requested a full tablet report, sending...
I20260620 14:53:20.773427 11375 ts_manager.cc:194] Registered new tserver with Master: 6456a06cf9e84ab280ab7ab2d3a5fccc (127.7.24.195:41871)
I20260620 14:53:20.774091 11375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:33217
I20260620 14:53:20.775087 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:53:20.784171 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.776932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34012 (local address 127.7.24.254:35571)
0620 14:53:20.777039 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:20.777042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:20.777206 (+ 164us) server_negotiation.cc:407] Connection header received
0620 14:53:20.777299 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:20.777301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:20.777335 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:20.777401 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:20.778062 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.778598 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.779435 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.779658 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.779798 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.781103 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:20.781107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:20.781110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:20.781134 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:20.782571 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:20.782972 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:20.782977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:20.782980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:20.783057 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:20.783326 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:20.783329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:20.783331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:20.783542 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:20.783673 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:20.783893 (+ 220us) server_negotiation.cc:299] Negotiation successful
0620 14:53:20.784005 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 14:53:20.786985 11375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:20.790124 11375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:20.804946 11847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.799488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36240 (local address 127.7.24.195:41871)
0620 14:53:20.800031 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:20.800041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:20.800066 (+ 25us) server_negotiation.cc:407] Connection header received
0620 14:53:20.800160 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:20.800167 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:20.800361 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:20.800513 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:20.800808 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.801773 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.803424 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.804185 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.804491 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.804629 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:20.804682 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 14:53:20.804790 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":131,"threads_started":1}
I20260620 14:53:20.804940 11843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.798732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:41871 (local address 127.0.0.1:36240)
0620 14:53:20.799708 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:20.799723 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:20.799797 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:20.800511 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:20.800517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:20.800540 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:20.800678 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.800684 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.801887 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.801892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.803296 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.803306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.804333 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.804336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.804454 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.804535 (+ 81us) client_negotiation.cc:769] Sending connection context
0620 14:53:20.804624 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 14:53:20.804768 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"mutex_wait_us":101,"thread_start_us":96,"threads_started":1}
I20260620 14:53:20.805222 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.798673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38147 (local address 127.0.0.1:39642)
0620 14:53:20.799482 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:20.799531 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:20.799659 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:20.800678 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:20.800681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:20.800696 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:20.800812 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.800818 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.802204 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.802207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.803370 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.803379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.804272 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.804279 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.804477 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.804522 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 14:53:20.804633 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 14:53:20.804884 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":164,"thread_start_us":99,"threads_started":1}
I20260620 14:53:20.805505 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.798801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39642 (local address 127.7.24.194:38147)
0620 14:53:20.800004 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:20.800015 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:20.800035 (+ 20us) server_negotiation.cc:407] Connection header received
0620 14:53:20.800126 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:20.800133 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:20.800353 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:20.800501 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:20.800927 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.802074 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.803494 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.804135 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.804263 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.804417 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:20.804740 (+ 323us) server_negotiation.cc:299] Negotiation successful
0620 14:53:20.805027 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1125,"thread_start_us":156,"threads_started":1}
I20260620 14:53:20.805644 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.799332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54812 (local address 127.7.24.193:43627)
0620 14:53:20.799577 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:20.799585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:20.800815 (+ 1230us) server_negotiation.cc:407] Connection header received
0620 14:53:20.801072 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:20.801078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:20.801271 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:20.801420 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:20.801634 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.802652 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.803969 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.804773 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.804870 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.804991 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:20.805369 (+ 378us) server_negotiation.cc:299] Negotiation successful
0620 14:53:20.805477 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":60,"threads_started":1}
I20260620 14:53:20.806082 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.799004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:43627 (local address 127.0.0.1:54812)
0620 14:53:20.800698 (+ 1694us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:20.800709 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:20.800887 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:20.801413 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:20.801416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:20.801434 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:20.801540 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.801546 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.802790 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.802793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.803842 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.803852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.805053 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.805057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.805222 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.805245 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 14:53:20.805845 (+ 600us) client_negotiation.cc:240] Negotiation successful
0620 14:53:20.805947 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1643,"thread_start_us":68,"threads_started":1}
I20260620 14:53:20.808187 11632 tablet_service.cc:1467] Processing CreateTablet for tablet e2926c10eb614714952e497ef354fc1e (DEFAULT_TABLE table=test-table [id=7be2b3470bbf44988aac8e5314d4a0cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:20.808368 11499 tablet_service.cc:1467] Processing CreateTablet for tablet e2926c10eb614714952e497ef354fc1e (DEFAULT_TABLE table=test-table [id=7be2b3470bbf44988aac8e5314d4a0cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:20.808342 11765 tablet_service.cc:1467] Processing CreateTablet for tablet e2926c10eb614714952e497ef354fc1e (DEFAULT_TABLE table=test-table [id=7be2b3470bbf44988aac8e5314d4a0cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:20.809267 11632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2926c10eb614714952e497ef354fc1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:20.809382 11765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2926c10eb614714952e497ef354fc1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:20.809378 11499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2926c10eb614714952e497ef354fc1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:20.816018 11848 tablet_bootstrap.cc:492] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc: Bootstrap starting.
I20260620 14:53:20.817196 11849 tablet_bootstrap.cc:492] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28: Bootstrap starting.
I20260620 14:53:20.818167 11848 tablet_bootstrap.cc:654] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:20.818859 11850 tablet_bootstrap.cc:492] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc: Bootstrap starting.
I20260620 14:53:20.819005 11849 tablet_bootstrap.cc:654] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:20.819442 11848 log.cc:824] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:20.819957 11849 log.cc:824] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:20.820931 11850 tablet_bootstrap.cc:654] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:20.821862 11849 tablet_bootstrap.cc:492] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28: No bootstrap required, opened a new log
I20260620 14:53:20.821913 11848 tablet_bootstrap.cc:492] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc: No bootstrap required, opened a new log
I20260620 14:53:20.822009 11850 log.cc:824] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:20.822085 11848 ts_tablet_manager.cc:1397] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 14:53:20.822074 11849 ts_tablet_manager.cc:1397] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 14:53:20.824010 11850 tablet_bootstrap.cc:492] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc: No bootstrap required, opened a new log
I20260620 14:53:20.824208 11850 ts_tablet_manager.cc:1397] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:53:20.825284 11849 raft_consensus.cc:348] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38147 } } peers { permanent_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41871 } } peers { permanent_uuid: "77f07c543f074011955b74d4c2591d28" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43627 } }
I20260620 14:53:20.825523 11849 raft_consensus.cc:374] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:20.825574 11849 raft_consensus.cc:729] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77f07c543f074011955b74d4c2591d28, State: Initialized, Role: FOLLOWER
I20260620 14:53:20.825917 11848 raft_consensus.cc:348] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38147 } } peers { permanent_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41871 } } peers { permanent_uuid: "77f07c543f074011955b74d4c2591d28" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43627 } }
I20260620 14:53:20.826047 11849 consensus_queue.cc:260] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38147 } } peers { permanent_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41871 } } peers { permanent_uuid: "77f07c543f074011955b74d4c2591d28" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43627 } }
I20260620 14:53:20.826167 11848 raft_consensus.cc:374] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:20.826223 11848 raft_consensus.cc:729] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c8d094cdbd34d4c89ea64a1785bffdc, State: Initialized, Role: FOLLOWER
I20260620 14:53:20.826622 11848 consensus_queue.cc:260] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38147 } } peers { permanent_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41871 } } peers { permanent_uuid: "77f07c543f074011955b74d4c2591d28" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43627 } }
I20260620 14:53:20.827030 11849 ts_tablet_manager.cc:1428] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 14:53:20.827359 11698 heartbeater.cc:503] Master 127.7.24.254:35571 was elected leader, sending a full tablet report...
I20260620 14:53:20.827519 11848 ts_tablet_manager.cc:1428] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 14:53:20.827955 11850 raft_consensus.cc:348] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38147 } } peers { permanent_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41871 } } peers { permanent_uuid: "77f07c543f074011955b74d4c2591d28" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43627 } }
I20260620 14:53:20.828132 11850 raft_consensus.cc:374] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:20.828169 11850 raft_consensus.cc:729] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6456a06cf9e84ab280ab7ab2d3a5fccc, State: Initialized, Role: FOLLOWER
I20260620 14:53:20.828425 11565 heartbeater.cc:503] Master 127.7.24.254:35571 was elected leader, sending a full tablet report...
I20260620 14:53:20.828580 11850 consensus_queue.cc:260] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38147 } } peers { permanent_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41871 } } peers { permanent_uuid: "77f07c543f074011955b74d4c2591d28" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43627 } }
I20260620 14:53:20.829207 11831 heartbeater.cc:503] Master 127.7.24.254:35571 was elected leader, sending a full tablet report...
I20260620 14:53:20.829468 11850 ts_tablet_manager.cc:1428] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 14:53:20.886332 11854 raft_consensus.cc:482] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:20.886590 11854 raft_consensus.cc:504] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38147 } } peers { permanent_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41871 } } peers { permanent_uuid: "77f07c543f074011955b74d4c2591d28" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43627 } }
I20260620 14:53:20.887719 11854 leader_election.cc:288] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c8d094cdbd34d4c89ea64a1785bffdc (127.7.24.194:38147), 6456a06cf9e84ab280ab7ab2d3a5fccc (127.7.24.195:41871)
I20260620 14:53:20.891959 11847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.888170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:53411 (local address 127.7.24.195:41871)
0620 14:53:20.888309 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:20.888313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:20.888625 (+ 312us) server_negotiation.cc:407] Connection header received
0620 14:53:20.888831 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:20.888834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:20.888883 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:20.888970 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:20.889248 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.889818 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.890984 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.891386 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.891511 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.891566 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:20.891750 (+ 184us) server_negotiation.cc:299] Negotiation successful
0620 14:53:20.891820 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 14:53:20.891975 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.888189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:46693 (local address 127.7.24.194:38147)
0620 14:53:20.888339 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:20.888343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:20.888569 (+ 226us) server_negotiation.cc:407] Connection header received
0620 14:53:20.888798 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:20.888802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:20.888852 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:20.888929 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:20.889234 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.889821 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.890988 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.891402 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.891523 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.891579 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:20.891776 (+ 197us) server_negotiation.cc:299] Negotiation successful
0620 14:53:20.891848 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 14:53:20.892005 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.888031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38147 (local address 127.7.24.193:46693)
0620 14:53:20.888468 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:20.888481 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:20.888685 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:20.888951 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:20.888954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:20.888970 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:20.889125 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.889131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.889927 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.889933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.890872 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.890882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.891523 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.891527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.891657 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.891676 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 14:53:20.891760 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 14:53:20.891841 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":30,"spinlock_wait_cycles":37888,"thread_start_us":133,"threads_started":1}
I20260620 14:53:20.892196 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.888031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:41871 (local address 127.7.24.193:53411)
0620 14:53:20.888521 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:20.888533 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:20.888687 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:20.888980 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:20.888983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:20.888999 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:20.889141 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.889145 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.889922 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.889928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.890866 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:20.890875 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.891508 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.891511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:20.891613 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.891636 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 14:53:20.891723 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 14:53:20.891811 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":135,"spinlock_wait_cycles":2688,"thread_start_us":134,"threads_started":1}
I20260620 14:53:20.892628 11652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2926c10eb614714952e497ef354fc1e" candidate_uuid: "77f07c543f074011955b74d4c2591d28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" is_pre_election: true
I20260620 14:53:20.892690 11785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2926c10eb614714952e497ef354fc1e" candidate_uuid: "77f07c543f074011955b74d4c2591d28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" is_pre_election: true
I20260620 14:53:20.892947 11652 raft_consensus.cc:2454] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77f07c543f074011955b74d4c2591d28 in term 0.
I20260620 14:53:20.892968 11785 raft_consensus.cc:2454] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77f07c543f074011955b74d4c2591d28 in term 0.
I20260620 14:53:20.893435 11452 leader_election.cc:302] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6456a06cf9e84ab280ab7ab2d3a5fccc, 77f07c543f074011955b74d4c2591d28; no voters:
I20260620 14:53:20.893716 11854 raft_consensus.cc:2781] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:20.893810 11854 raft_consensus.cc:482] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:20.893854 11854 raft_consensus.cc:3037] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:20.894960 11854 raft_consensus.cc:504] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38147 } } peers { permanent_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41871 } } peers { permanent_uuid: "77f07c543f074011955b74d4c2591d28" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43627 } }
I20260620 14:53:20.895382 11854 leader_election.cc:288] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [CANDIDATE]: Term 1 election: Requested vote from peers 0c8d094cdbd34d4c89ea64a1785bffdc (127.7.24.194:38147), 6456a06cf9e84ab280ab7ab2d3a5fccc (127.7.24.195:41871)
I20260620 14:53:20.895783 11652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2926c10eb614714952e497ef354fc1e" candidate_uuid: "77f07c543f074011955b74d4c2591d28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc"
I20260620 14:53:20.895920 11652 raft_consensus.cc:3037] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:20.896040 11785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2926c10eb614714952e497ef354fc1e" candidate_uuid: "77f07c543f074011955b74d4c2591d28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc"
I20260620 14:53:20.896152 11785 raft_consensus.cc:3037] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:20.897287 11652 raft_consensus.cc:2454] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77f07c543f074011955b74d4c2591d28 in term 1.
I20260620 14:53:20.897416 11785 raft_consensus.cc:2454] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77f07c543f074011955b74d4c2591d28 in term 1.
I20260620 14:53:20.897610 11453 leader_election.cc:302] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c8d094cdbd34d4c89ea64a1785bffdc, 77f07c543f074011955b74d4c2591d28; no voters:
I20260620 14:53:20.897778 11854 raft_consensus.cc:2781] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:20.898000 11854 raft_consensus.cc:686] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [term 1 LEADER]: Becoming Leader. State: Replica: 77f07c543f074011955b74d4c2591d28, State: Running, Role: LEADER
I20260620 14:53:20.898339 11854 consensus_queue.cc:237] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38147 } } peers { permanent_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41871 } } peers { permanent_uuid: "77f07c543f074011955b74d4c2591d28" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43627 } }
I20260620 14:53:20.901702 11373 catalog_manager.cc:5495] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77f07c543f074011955b74d4c2591d28 (127.7.24.193). New cstate: current_term: 1 leader_uuid: "77f07c543f074011955b74d4c2591d28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77f07c543f074011955b74d4c2591d28" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43627 } health_report { overall_health: HEALTHY } } }
I20260620 14:53:20.928732 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.925036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.7.24.193:43627)
0620 14:53:20.925163 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:20.925166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:20.925179 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:53:20.925290 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:20.925292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:20.925345 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:20.925433 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:20.925648 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.926136 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.927186 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:20.927417 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:20.927509 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:20.927890 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:20.928346 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:20.928445 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:20.928554 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 14:53:20.928610 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:53:20.934824 11785 raft_consensus.cc:1261] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc [term 1 FOLLOWER]: Refusing update from remote peer 77f07c543f074011955b74d4c2591d28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:20.934824 11652 raft_consensus.cc:1261] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc [term 1 FOLLOWER]: Refusing update from remote peer 77f07c543f074011955b74d4c2591d28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:20.935576 11859 consensus_queue.cc:1035] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6456a06cf9e84ab280ab7ab2d3a5fccc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:20.935774 11854 consensus_queue.cc:1035] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c8d094cdbd34d4c89ea64a1785bffdc" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:20.945730 11864 mvcc.cc:205] Tried to move back new op lower bound from 7298937655021727744 to 7298937654882594816. Current Snapshot: MvccSnapshot[applied={T|T < 7298937655021727744}]
I20260620 14:53:20.951972 11374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:20.952158 11374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:20.954778 11374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 347227bf0def4f72ac63a13143fceee2: Sending DeleteTablet for 3 replicas of tablet e2926c10eb614714952e497ef354fc1e
I20260620 14:53:20.955744 11632 tablet_service.cc:1514] Processing DeleteTablet for tablet e2926c10eb614714952e497ef354fc1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:20 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:39642
I20260620 14:53:20.955744 11765 tablet_service.cc:1514] Processing DeleteTablet for tablet e2926c10eb614714952e497ef354fc1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:20 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:36240
I20260620 14:53:20.956307 11872 tablet_replica.cc:330] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc: stopping tablet replica
I20260620 14:53:20.956434 11499 tablet_service.cc:1514] Processing DeleteTablet for tablet e2926c10eb614714952e497ef354fc1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:20 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54812
I20260620 14:53:20.956642 11873 tablet_replica.cc:330] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc: stopping tablet replica
I20260620 14:53:20.956902 11874 tablet_replica.cc:330] T e2926c10eb614714952e497ef354fc1e P 77f07c543f074011955b74d4c2591d28: stopping tablet replica
I20260620 14:53:20.956897 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 11437
I20260620 14:53:20.958405 11872 raft_consensus.cc:2229] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:20.958636 11873 raft_consensus.cc:2229] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:20.959301 11873 raft_consensus.cc:2258] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:20.959813 11872 raft_consensus.cc:2258] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:20.960826 11873 ts_tablet_manager.cc:1905] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:20.961231 11872 ts_tablet_manager.cc:1905] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:20.963172 11873 ts_tablet_manager.cc:1918] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:20.963302 11873 log.cc:1196] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/e2926c10eb614714952e497ef354fc1e
I20260620 14:53:20.963621 11873 ts_tablet_manager.cc:1939] T e2926c10eb614714952e497ef354fc1e P 6456a06cf9e84ab280ab7ab2d3a5fccc: Deleting consensus metadata
I20260620 14:53:20.963732 11872 ts_tablet_manager.cc:1918] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:20.963825 11872 log.cc:1196] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/e2926c10eb614714952e497ef354fc1e
I20260620 14:53:20.964123 11872 ts_tablet_manager.cc:1939] T e2926c10eb614714952e497ef354fc1e P 0c8d094cdbd34d4c89ea64a1785bffdc: Deleting consensus metadata
W20260620 14:53:20.964756 11361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:43627 (error 108)
I20260620 14:53:20.965013 11359 catalog_manager.cc:4841] TS 6456a06cf9e84ab280ab7ab2d3a5fccc (127.7.24.195:41871): tablet e2926c10eb614714952e497ef354fc1e (table test-table [id=7be2b3470bbf44988aac8e5314d4a0cb]) successfully deleted
I20260620 14:53:20.965235 11360 catalog_manager.cc:4841] TS 0c8d094cdbd34d4c89ea64a1785bffdc (127.7.24.194:38147): tablet e2926c10eb614714952e497ef354fc1e (table test-table [id=7be2b3470bbf44988aac8e5314d4a0cb]) successfully deleted
I20260620 14:53:20.965777 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:20.965522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:43627 (local address 127.0.0.1:54834)
0620 14:53:20.965643 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:20.965704 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:43627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260620 14:53:20.965899 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 11570
W20260620 14:53:20.966006 11361 catalog_manager.cc:4584] TS 77f07c543f074011955b74d4c2591d28 (127.7.24.193:43627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e2926c10eb614714952e497ef354fc1e: Network error: Client connection negotiation failed: client connection to 127.7.24.193:43627: connect: Connection refused (error 111)
I20260620 14:53:20.971736 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 11703
I20260620 14:53:20.977577 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 11344
2026-06-20T14:53:20Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2319 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-20T14:53:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:21Z Disabled control of system clock
I20260620 14:53:21.008095 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:43721
--webserver_interface=127.7.24.254
--webserver_port=0
--builtin_ntp_servers=127.7.24.212:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:43721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 14:53:21.117107 11879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:21.117383 11879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:21.117421 11879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:21.121505 11879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:21.121567 11879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:21.121583 11879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:21.121600 11879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:21.121615 11879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:21.126416 11879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:43721
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:43721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:21.127476 11879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:21.128783 11879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:21.137266 11884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:21.137328 11885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:21.137267 11887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:21.137563 11879 server_base.cc:1034] running on GCE node
I20260620 14:53:21.138352 11879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:21.139498 11879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:21.140671 11879 hybrid_clock.cc:648] HybridClock initialized: now 1781967201140649 us; error 31 us; skew 500 ppm
I20260620 14:53:21.142577 11879 webserver.cc:460] Webserver started at http://127.7.24.254:44443/ using document root <none> and password file <none>
I20260620 14:53:21.143210 11879 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:21.143291 11879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:21.143513 11879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:21.145539 11879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "cbbbf7d5dbc9406c8cc9f93eb4501dcb"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "35caf78349d7dedf0c23cda43c684dfe"
server_key_iv: "d396013de3b0c204b932640e336d514f"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.146085 11879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "cbbbf7d5dbc9406c8cc9f93eb4501dcb"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "35caf78349d7dedf0c23cda43c684dfe"
server_key_iv: "d396013de3b0c204b932640e336d514f"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.149899 11879 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 14:53:21.152297 11893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:21.153606 11879 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:21.153748 11879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "cbbbf7d5dbc9406c8cc9f93eb4501dcb"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "35caf78349d7dedf0c23cda43c684dfe"
server_key_iv: "d396013de3b0c204b932640e336d514f"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.153868 11879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:21.158547 11879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:21.159168 11879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:21.159376 11879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:21.166500 11879 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:43721
I20260620 14:53:21.166554 11945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:43721 every 8 connection(s)
I20260620 14:53:21.167529 11879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:21.170981 11946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:21.173686 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 11879
I20260620 14:53:21.173808 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:21.174115 7267 external_mini_cluster.cc:1428] Setting key 1fe0dda963fdf4f52609e78e164267d4
I20260620 14:53:21.177829 11946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb: Bootstrap starting.
I20260620 14:53:21.180619 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.175703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52866 (local address 127.7.24.254:43721)
0620 14:53:21.176453 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:21.176475 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:21.176525 (+ 50us) server_negotiation.cc:407] Connection header received
0620 14:53:21.177379 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:21.177401 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:21.177853 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:21.178240 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:21.179064 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:21.179072 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:21.179090 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:21.179106 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:21.179189 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:21.179213 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:21.179353 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:21.179422 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 14:53:21.179663 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":126,"threads_started":1}
I20260620 14:53:21.180668 11946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:21.181420 11946 log.cc:824] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:21.183845 11946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb: No bootstrap required, opened a new log
I20260620 14:53:21.188560 11946 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbf7d5dbc9406c8cc9f93eb4501dcb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43721 } }
I20260620 14:53:21.188817 11946 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:21.188853 11946 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbbbf7d5dbc9406c8cc9f93eb4501dcb, State: Initialized, Role: FOLLOWER
I20260620 14:53:21.189337 11946 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbf7d5dbc9406c8cc9f93eb4501dcb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43721 } }
I20260620 14:53:21.189494 11946 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:21.189579 11946 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:21.189682 11946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:21.190824 11946 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbf7d5dbc9406c8cc9f93eb4501dcb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43721 } }
I20260620 14:53:21.191236 11946 leader_election.cc:302] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbbbf7d5dbc9406c8cc9f93eb4501dcb; no voters:
I20260620 14:53:21.191620 11946 leader_election.cc:288] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:21.191797 11951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:21.192044 11951 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [term 1 LEADER]: Becoming Leader. State: Replica: cbbbf7d5dbc9406c8cc9f93eb4501dcb, State: Running, Role: LEADER
I20260620 14:53:21.192360 11951 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbf7d5dbc9406c8cc9f93eb4501dcb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43721 } }
I20260620 14:53:21.192930 11946 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:21.194026 11952 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbbbf7d5dbc9406c8cc9f93eb4501dcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbf7d5dbc9406c8cc9f93eb4501dcb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43721 } } }
I20260620 14:53:21.194182 11952 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:21.194510 11953 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbbbf7d5dbc9406c8cc9f93eb4501dcb. Latest consensus state: current_term: 1 leader_uuid: "cbbbf7d5dbc9406c8cc9f93eb4501dcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbf7d5dbc9406c8cc9f93eb4501dcb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 43721 } } }
I20260620 14:53:21.194608 11953 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:21.194957 11960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:21.198215 11960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:21.204602 11960 catalog_manager.cc:1347] Generated new cluster ID: f2d1fe82f70d4544ba6d098bdfbac651
I20260620 14:53:21.204701 11960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:21.224048 11960 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:21.224416 11960 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:21.232230 11960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb: Generated new TSK 0
I20260620 14:53:21.233136 11960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:53:21.235893 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43721
--builtin_ntp_servers=127.7.24.212:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 14:53:21.359241 11970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:21.359498 11970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:21.359539 11970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:21.363561 11970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:21.363615 11970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:21.363705 11970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:21.368433 11970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:21.369493 11970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:21.370872 11970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:21.380208 11978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:21.380218 11975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:21.380208 11976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:21.380654 11970 server_base.cc:1034] running on GCE node
I20260620 14:53:21.381045 11970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:21.381690 11970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:21.382892 11970 hybrid_clock.cc:648] HybridClock initialized: now 1781967201382859 us; error 45 us; skew 500 ppm
I20260620 14:53:21.384692 11970 webserver.cc:460] Webserver started at http://127.7.24.193:40059/ using document root <none> and password file <none>
I20260620 14:53:21.385298 11970 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:21.385388 11970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:21.385613 11970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:21.387723 11970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "a5b0aba16a784ec3b37eff0bace9c750"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "ce262930351844bd11e939052a67a61e"
server_key_iv: "70ab8fd4d2951faa6d0b82a2d261933e"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.388284 11970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "a5b0aba16a784ec3b37eff0bace9c750"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "ce262930351844bd11e939052a67a61e"
server_key_iv: "70ab8fd4d2951faa6d0b82a2d261933e"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.391993 11970 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 14:53:21.394555 11984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:21.395843 11970 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:21.395958 11970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "a5b0aba16a784ec3b37eff0bace9c750"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "ce262930351844bd11e939052a67a61e"
server_key_iv: "70ab8fd4d2951faa6d0b82a2d261933e"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.396070 11970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:21.400604 11970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:21.401208 11970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:21.401394 11970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:21.401926 11970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:21.402848 11970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:21.402913 11970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:21.402979 11970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:21.403023 11970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:21.412106 11970 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:33759
I20260620 14:53:21.412132 12097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:33759 every 8 connection(s)
I20260620 14:53:21.412982 11970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:21.419013 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.414707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:35959 (local address 127.7.24.254:43721)
0620 14:53:21.414851 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:21.414855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:21.415984 (+ 1129us) server_negotiation.cc:407] Connection header received
0620 14:53:21.417297 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:21.417300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:21.417345 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:21.417448 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:21.418513 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:21.418517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:21.418520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:21.418532 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:21.418554 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:21.418568 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:21.418691 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:21.418827 (+ 136us) server_negotiation.cc:299] Negotiation successful
0620 14:53:21.418873 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 14:53:21.419866 12100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.415142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:43721 (local address 127.7.24.193:35959)
0620 14:53:21.415838 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:21.415880 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:21.417071 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:21.417608 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:21.417615 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:21.417702 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:21.418234 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:21.418235 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:21.418273 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:21.418336 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:21.418674 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:21.418682 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:21.418687 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 14:53:21.418866 (+ 179us) client_negotiation.cc:240] Negotiation successful
0620 14:53:21.419100 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260620 14:53:21.421003 12098 heartbeater.cc:347] Connected to a master server at 127.7.24.254:43721
I20260620 14:53:21.421233 12098 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:21.421658 12098 heartbeater.cc:511] Master 127.7.24.254:43721 requested a full tablet report, sending...
I20260620 14:53:21.422901 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 11970
I20260620 14:53:21.422996 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:21.423291 7267 external_mini_cluster.cc:1428] Setting key e40c031a1f326e973bc3132f004d8c34
I20260620 14:53:21.423344 11910 ts_manager.cc:194] Registered new tserver with Master: a5b0aba16a784ec3b37eff0bace9c750 (127.7.24.193:33759)
I20260620 14:53:21.425385 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43721
--builtin_ntp_servers=127.7.24.212:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 14:53:21.538512 12101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:21.538801 12101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:21.538874 12101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:21.542925 12101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:21.542986 12101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:21.543043 12101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:21.547883 12101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:21.548964 12101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:21.550439 12101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:21.560144 12109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:21.560213 12107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:21.560161 12106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:21.560384 12101 server_base.cc:1034] running on GCE node
I20260620 14:53:21.560962 12101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:21.561635 12101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:21.562798 12101 hybrid_clock.cc:648] HybridClock initialized: now 1781967201562775 us; error 31 us; skew 500 ppm
I20260620 14:53:21.564734 12101 webserver.cc:460] Webserver started at http://127.7.24.194:44185/ using document root <none> and password file <none>
I20260620 14:53:21.565321 12101 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:21.565385 12101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:21.565627 12101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:21.567795 12101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "133012dfe80842df9b8c9864af5c1828"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "dbc4a92b10b4827c01fb7f4e9303354b"
server_key_iv: "ed41f03ff79ff7366d5c588bb91f2b8e"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.568365 12101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "133012dfe80842df9b8c9864af5c1828"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "dbc4a92b10b4827c01fb7f4e9303354b"
server_key_iv: "ed41f03ff79ff7366d5c588bb91f2b8e"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.572126 12101 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 14:53:21.574434 12115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:21.575759 12101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:21.575896 12101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "133012dfe80842df9b8c9864af5c1828"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "dbc4a92b10b4827c01fb7f4e9303354b"
server_key_iv: "ed41f03ff79ff7366d5c588bb91f2b8e"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.576000 12101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:21.580617 12101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:21.581198 12101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:21.581372 12101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:21.581897 12101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:21.582837 12101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:21.582883 12101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:21.582948 12101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:21.582985 12101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:21.591354 12101 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:36405
I20260620 14:53:21.591379 12228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:36405 every 8 connection(s)
I20260620 14:53:21.592226 12101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:21.598192 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.594139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:43565 (local address 127.7.24.254:43721)
0620 14:53:21.594299 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:21.594304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:21.595455 (+ 1151us) server_negotiation.cc:407] Connection header received
0620 14:53:21.596695 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:21.596697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:21.596741 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:21.596816 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:21.597826 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:21.597829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:21.597831 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:21.597842 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:21.597860 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:21.597867 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:21.597943 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:21.598050 (+ 107us) server_negotiation.cc:299] Negotiation successful
0620 14:53:21.598085 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 14:53:21.599092 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.594622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:43721 (local address 127.7.24.194:43565)
0620 14:53:21.595316 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:21.595355 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:21.596506 (+ 1151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:21.596964 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:21.596970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:21.597055 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:21.597575 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:21.597577 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:21.597614 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:21.597677 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:21.597937 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:21.597942 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:21.597946 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 14:53:21.598071 (+ 125us) client_negotiation.cc:240] Negotiation successful
0620 14:53:21.598251 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":138,"threads_started":1}
I20260620 14:53:21.600165 12229 heartbeater.cc:347] Connected to a master server at 127.7.24.254:43721
I20260620 14:53:21.600382 12229 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:21.600800 12229 heartbeater.cc:511] Master 127.7.24.254:43721 requested a full tablet report, sending...
I20260620 14:53:21.601254 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 12101
I20260620 14:53:21.601380 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:21.601658 7267 external_mini_cluster.cc:1428] Setting key f1ee83013a9ea8562bd15564b9291f61
I20260620 14:53:21.601945 11909 ts_manager.cc:194] Registered new tserver with Master: 133012dfe80842df9b8c9864af5c1828 (127.7.24.194:36405)
I20260620 14:53:21.603452 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43721
--builtin_ntp_servers=127.7.24.212:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 14:53:21.711328 12232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:21.711616 12232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:21.711691 12232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:21.715739 12232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:21.715821 12232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:21.715914 12232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:21.720642 12232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:43721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:21.721740 12232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:21.723119 12232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:21.732470 12237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:21.732477 12238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:21.732477 12240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:21.732628 12232 server_base.cc:1034] running on GCE node
I20260620 14:53:21.733139 12232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:21.733771 12232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:21.734966 12232 hybrid_clock.cc:648] HybridClock initialized: now 1781967201734951 us; error 33 us; skew 500 ppm
I20260620 14:53:21.736840 12232 webserver.cc:460] Webserver started at http://127.7.24.195:38969/ using document root <none> and password file <none>
I20260620 14:53:21.737401 12232 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:21.737488 12232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:21.737720 12232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:21.739795 12232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "7216c0f10096425f9ef797f9f9428c4f"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "67b7131b5d8f053d7d714eec52c87e52"
server_key_iv: "34af86f2494024a758c31fe3349ef122"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.740356 12232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "7216c0f10096425f9ef797f9f9428c4f"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "67b7131b5d8f053d7d714eec52c87e52"
server_key_iv: "34af86f2494024a758c31fe3349ef122"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.744096 12232 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 14:53:21.746941 12246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:21.748257 12232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:21.748400 12232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "7216c0f10096425f9ef797f9f9428c4f"
format_stamp: "Formatted at 2026-06-20 14:53:21 on dist-test-slave-sms9"
server_key: "67b7131b5d8f053d7d714eec52c87e52"
server_key_iv: "34af86f2494024a758c31fe3349ef122"
server_key_version: "encryptionkey@0"
I20260620 14:53:21.748514 12232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:21.753129 12232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:21.753731 12232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:21.753903 12232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:21.754427 12232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:21.755334 12232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:21.755402 12232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:21.755467 12232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:21.755513 12232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:21.764855 12232 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:43851
I20260620 14:53:21.764873 12359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:43851 every 8 connection(s)
I20260620 14:53:21.765774 12232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:21.768965 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 12232
I20260620 14:53:21.769086 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:21.769387 7267 external_mini_cluster.cc:1428] Setting key 4d9d393177a52f17575b64c678e25478
I20260620 14:53:21.772184 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.767602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:46567 (local address 127.7.24.254:43721)
0620 14:53:21.767751 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:21.767757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:21.768953 (+ 1196us) server_negotiation.cc:407] Connection header received
0620 14:53:21.770640 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:21.770643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:21.770680 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:21.770805 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:21.771795 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:21.771798 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:21.771800 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:21.771811 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:21.771832 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:21.771840 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:21.771921 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:21.772043 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 14:53:21.772080 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 14:53:21.773042 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.768066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:43721 (local address 127.7.24.195:46567)
0620 14:53:21.768764 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:21.768818 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:21.770397 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:21.770930 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:21.770937 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:21.771019 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:21.771548 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:21.771549 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:21.771588 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:21.771651 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:21.771928 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:21.771933 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:21.771937 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 14:53:21.772062 (+ 125us) client_negotiation.cc:240] Negotiation successful
0620 14:53:21.772244 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":116,"threads_started":1}
I20260620 14:53:21.774029 12360 heartbeater.cc:347] Connected to a master server at 127.7.24.254:43721
I20260620 14:53:21.774370 12360 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:21.774784 12360 heartbeater.cc:511] Master 127.7.24.254:43721 requested a full tablet report, sending...
I20260620 14:53:21.775730 11910 ts_manager.cc:194] Registered new tserver with Master: 7216c0f10096425f9ef797f9f9428c4f (127.7.24.195:43851)
I20260620 14:53:21.783196 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:53:21.786499 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.785020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.7.24.254:43721)
0620 14:53:21.785157 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:21.785163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:21.785331 (+ 168us) server_negotiation.cc:407] Connection header received
0620 14:53:21.785434 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:21.785436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:21.785470 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:21.785535 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:21.786128 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:21.786132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:21.786134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:21.786144 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:21.786161 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:21.786168 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:21.786244 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:21.786292 (+ 48us) server_negotiation.cc:299] Negotiation successful
0620 14:53:21.786324 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 14:53:21.789105 11910 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:21.792192 11910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:21.803736 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.801817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36488 (local address 127.7.24.194:36405)
0620 14:53:21.802202 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:21.802214 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:21.802235 (+ 21us) server_negotiation.cc:407] Connection header received
0620 14:53:21.802295 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:21.802299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:21.802452 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:21.802619 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:21.803196 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:21.803202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:21.803212 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:21.803222 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:21.803267 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:21.803284 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:21.803439 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:21.803513 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 14:53:21.803582 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":106,"threads_started":1}
I20260620 14:53:21.803704 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.800433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:43851 (local address 127.0.0.1:58168)
0620 14:53:21.800916 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:21.800963 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:21.801064 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:21.802276 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:21.802280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:21.802300 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:21.802881 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:21.802884 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:21.802906 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:21.802977 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:21.803343 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:21.803355 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:21.803363 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 14:53:21.803474 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 14:53:21.803544 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":69,"threads_started":1}
I20260620 14:53:21.803709 12372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.800568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:36405 (local address 127.0.0.1:36488)
0620 14:53:21.800925 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:21.800968 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:21.801058 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:21.802620 (+ 1562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:21.802625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:21.802654 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:21.803055 (+ 401us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:21.803057 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:21.803068 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:21.803095 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:21.803417 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:21.803420 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:21.803423 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 14:53:21.803509 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 14:53:21.803549 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":107,"threads_started":1}
I20260620 14:53:21.804229 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.801088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33759 (local address 127.0.0.1:34496)
0620 14:53:21.801298 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:21.801314 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:21.801383 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:21.802020 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:21.802028 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:21.802104 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:21.803084 (+ 980us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:21.803086 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:21.803096 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:21.803119 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:21.804028 (+ 909us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:21.804033 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:21.804038 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 14:53:21.804092 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 14:53:21.804120 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260620 14:53:21.804493 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.801415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.7.24.195:43851)
0620 14:53:21.801777 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:21.801790 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:21.801813 (+ 23us) server_negotiation.cc:407] Connection header received
0620 14:53:21.801864 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:21.801869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:21.802031 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:21.802167 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:21.803134 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:21.803142 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:21.803158 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:21.803172 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:21.803216 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:21.803250 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:21.803357 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:21.804028 (+ 671us) server_negotiation.cc:299] Negotiation successful
0620 14:53:21.804097 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":162,"threads_started":1}
I20260620 14:53:21.804713 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.801229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34496 (local address 127.7.24.193:33759)
0620 14:53:21.801525 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:21.801538 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:21.801563 (+ 25us) server_negotiation.cc:407] Connection header received
0620 14:53:21.801617 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:21.801639 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:21.801848 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:21.803258 (+ 1410us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:21.803817 (+ 559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:21.803823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:21.803833 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:21.803843 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:21.803880 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:21.803904 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:21.804416 (+ 512us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:21.804482 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 14:53:21.804547 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
I20260620 14:53:21.806991 12294 tablet_service.cc:1467] Processing CreateTablet for tablet 1831f80cc7a142469d693e2dd40e48b3 (DEFAULT_TABLE table=test-table [id=7ba3141bb1584918a19cd76800e0a15c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:21.806973 12163 tablet_service.cc:1467] Processing CreateTablet for tablet 1831f80cc7a142469d693e2dd40e48b3 (DEFAULT_TABLE table=test-table [id=7ba3141bb1584918a19cd76800e0a15c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:21.808025 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1831f80cc7a142469d693e2dd40e48b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:21.808032 12294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1831f80cc7a142469d693e2dd40e48b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:21.808990 12032 tablet_service.cc:1467] Processing CreateTablet for tablet 1831f80cc7a142469d693e2dd40e48b3 (DEFAULT_TABLE table=test-table [id=7ba3141bb1584918a19cd76800e0a15c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:21.809909 12032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1831f80cc7a142469d693e2dd40e48b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:21.813901 12377 tablet_bootstrap.cc:492] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f: Bootstrap starting.
I20260620 14:53:21.815400 12378 tablet_bootstrap.cc:492] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: Bootstrap starting.
I20260620 14:53:21.816102 12377 tablet_bootstrap.cc:654] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:21.817111 12377 log.cc:824] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:21.817294 12378 tablet_bootstrap.cc:654] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:21.817895 12378 log.cc:824] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:21.818558 12379 tablet_bootstrap.cc:492] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750: Bootstrap starting.
I20260620 14:53:21.818922 12377 tablet_bootstrap.cc:492] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f: No bootstrap required, opened a new log
I20260620 14:53:21.819100 12377 ts_tablet_manager.cc:1397] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 14:53:21.819621 12378 tablet_bootstrap.cc:492] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: No bootstrap required, opened a new log
I20260620 14:53:21.819789 12378 ts_tablet_manager.cc:1397] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 14:53:21.820226 12379 tablet_bootstrap.cc:654] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:21.820828 12379 log.cc:824] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:21.822517 12379 tablet_bootstrap.cc:492] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750: No bootstrap required, opened a new log
I20260620 14:53:21.822710 12379 ts_tablet_manager.cc:1397] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260620 14:53:21.823662 12377 raft_consensus.cc:348] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7216c0f10096425f9ef797f9f9428c4f" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43851 } } peers { permanent_uuid: "133012dfe80842df9b8c9864af5c1828" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36405 } } peers { permanent_uuid: "a5b0aba16a784ec3b37eff0bace9c750" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33759 } }
I20260620 14:53:21.823966 12377 raft_consensus.cc:374] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:21.824026 12377 raft_consensus.cc:729] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7216c0f10096425f9ef797f9f9428c4f, State: Initialized, Role: FOLLOWER
I20260620 14:53:21.824016 12378 raft_consensus.cc:348] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7216c0f10096425f9ef797f9f9428c4f" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43851 } } peers { permanent_uuid: "133012dfe80842df9b8c9864af5c1828" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36405 } } peers { permanent_uuid: "a5b0aba16a784ec3b37eff0bace9c750" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33759 } }
I20260620 14:53:21.824236 12378 raft_consensus.cc:374] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:21.824292 12378 raft_consensus.cc:729] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 133012dfe80842df9b8c9864af5c1828, State: Initialized, Role: FOLLOWER
I20260620 14:53:21.824541 12377 consensus_queue.cc:260] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7216c0f10096425f9ef797f9f9428c4f" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43851 } } peers { permanent_uuid: "133012dfe80842df9b8c9864af5c1828" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36405 } } peers { permanent_uuid: "a5b0aba16a784ec3b37eff0bace9c750" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33759 } }
I20260620 14:53:21.824617 12378 consensus_queue.cc:260] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7216c0f10096425f9ef797f9f9428c4f" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43851 } } peers { permanent_uuid: "133012dfe80842df9b8c9864af5c1828" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36405 } } peers { permanent_uuid: "a5b0aba16a784ec3b37eff0bace9c750" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33759 } }
I20260620 14:53:21.825250 12360 heartbeater.cc:503] Master 127.7.24.254:43721 was elected leader, sending a full tablet report...
I20260620 14:53:21.825517 12378 ts_tablet_manager.cc:1428] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 14:53:21.825834 12229 heartbeater.cc:503] Master 127.7.24.254:43721 was elected leader, sending a full tablet report...
I20260620 14:53:21.826519 12377 ts_tablet_manager.cc:1428] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260620 14:53:21.826596 12379 raft_consensus.cc:348] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7216c0f10096425f9ef797f9f9428c4f" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43851 } } peers { permanent_uuid: "133012dfe80842df9b8c9864af5c1828" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36405 } } peers { permanent_uuid: "a5b0aba16a784ec3b37eff0bace9c750" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33759 } }
I20260620 14:53:21.826838 12379 raft_consensus.cc:374] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:21.826894 12379 raft_consensus.cc:729] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5b0aba16a784ec3b37eff0bace9c750, State: Initialized, Role: FOLLOWER
I20260620 14:53:21.827291 12379 consensus_queue.cc:260] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7216c0f10096425f9ef797f9f9428c4f" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43851 } } peers { permanent_uuid: "133012dfe80842df9b8c9864af5c1828" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36405 } } peers { permanent_uuid: "a5b0aba16a784ec3b37eff0bace9c750" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33759 } }
I20260620 14:53:21.827891 12098 heartbeater.cc:503] Master 127.7.24.254:43721 was elected leader, sending a full tablet report...
I20260620 14:53:21.829092 12379 ts_tablet_manager.cc:1428] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260620 14:53:21.843204 12230 tablet.cc:2369] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:53:21.899300 12384 raft_consensus.cc:482] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:21.899535 12384 raft_consensus.cc:504] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7216c0f10096425f9ef797f9f9428c4f" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43851 } } peers { permanent_uuid: "133012dfe80842df9b8c9864af5c1828" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36405 } } peers { permanent_uuid: "a5b0aba16a784ec3b37eff0bace9c750" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33759 } }
I20260620 14:53:21.900769 12384 leader_election.cc:288] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7216c0f10096425f9ef797f9f9428c4f (127.7.24.195:43851), a5b0aba16a784ec3b37eff0bace9c750 (127.7.24.193:33759)
I20260620 14:53:21.903018 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.901119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:42069 (local address 127.7.24.193:33759)
0620 14:53:21.901231 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:21.901235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:21.901601 (+ 366us) server_negotiation.cc:407] Connection header received
0620 14:53:21.901701 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:21.901705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:21.901745 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:21.901836 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:21.902580 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:21.902584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:21.902586 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:21.902639 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:21.902658 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:21.902667 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:21.902752 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:21.902852 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 14:53:21.902894 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 14:53:21.903062 12386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.901015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33759 (local address 127.7.24.194:42069)
0620 14:53:21.901526 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:21.901540 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:21.901623 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:21.901844 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:21.901846 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:21.901860 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:21.902397 (+ 537us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:21.902399 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:21.902409 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:21.902436 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:21.902745 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:21.902747 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:21.902750 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 14:53:21.902844 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 14:53:21.902884 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":166,"thread_start_us":119,"threads_started":1}
I20260620 14:53:21.903079 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.901090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:49797 (local address 127.7.24.195:43851)
0620 14:53:21.901225 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:21.901240 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:21.901258 (+ 18us) server_negotiation.cc:407] Connection header received
0620 14:53:21.901375 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:21.901379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:21.901418 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:21.901506 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:21.902395 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:21.902399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:21.902403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:21.902416 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:21.902434 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:21.902444 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:21.902573 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:21.902911 (+ 338us) server_negotiation.cc:299] Negotiation successful
0620 14:53:21.902956 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 14:53:21.903072 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.901011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:43851 (local address 127.7.24.194:49797)
0620 14:53:21.901156 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:21.901174 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:21.901266 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:21.901546 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:21.901552 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:21.901567 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:21.902178 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:21.902181 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:21.902195 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:21.902232 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:21.902792 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:21.902795 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:21.902797 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 14:53:21.902901 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 14:53:21.902942 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260620 14:53:21.903883 12052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1831f80cc7a142469d693e2dd40e48b3" candidate_uuid: "133012dfe80842df9b8c9864af5c1828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5b0aba16a784ec3b37eff0bace9c750" is_pre_election: true
I20260620 14:53:21.904187 12314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1831f80cc7a142469d693e2dd40e48b3" candidate_uuid: "133012dfe80842df9b8c9864af5c1828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7216c0f10096425f9ef797f9f9428c4f" is_pre_election: true
I20260620 14:53:21.904222 12052 raft_consensus.cc:2454] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 133012dfe80842df9b8c9864af5c1828 in term 0.
I20260620 14:53:21.904441 12314 raft_consensus.cc:2454] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 133012dfe80842df9b8c9864af5c1828 in term 0.
I20260620 14:53:21.904754 12116 leader_election.cc:302] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 133012dfe80842df9b8c9864af5c1828, a5b0aba16a784ec3b37eff0bace9c750; no voters:
I20260620 14:53:21.905052 12384 raft_consensus.cc:2781] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:21.905150 12384 raft_consensus.cc:482] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:21.905208 12384 raft_consensus.cc:3037] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:21.906210 12384 raft_consensus.cc:504] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7216c0f10096425f9ef797f9f9428c4f" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43851 } } peers { permanent_uuid: "133012dfe80842df9b8c9864af5c1828" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36405 } } peers { permanent_uuid: "a5b0aba16a784ec3b37eff0bace9c750" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33759 } }
I20260620 14:53:21.906597 12384 leader_election.cc:288] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [CANDIDATE]: Term 1 election: Requested vote from peers 7216c0f10096425f9ef797f9f9428c4f (127.7.24.195:43851), a5b0aba16a784ec3b37eff0bace9c750 (127.7.24.193:33759)
I20260620 14:53:21.906963 12052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1831f80cc7a142469d693e2dd40e48b3" candidate_uuid: "133012dfe80842df9b8c9864af5c1828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5b0aba16a784ec3b37eff0bace9c750"
I20260620 14:53:21.907107 12314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1831f80cc7a142469d693e2dd40e48b3" candidate_uuid: "133012dfe80842df9b8c9864af5c1828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7216c0f10096425f9ef797f9f9428c4f"
I20260620 14:53:21.907111 12052 raft_consensus.cc:3037] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:21.907222 12314 raft_consensus.cc:3037] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:21.908207 12314 raft_consensus.cc:2454] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 133012dfe80842df9b8c9864af5c1828 in term 1.
I20260620 14:53:21.908241 12052 raft_consensus.cc:2454] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 133012dfe80842df9b8c9864af5c1828 in term 1.
I20260620 14:53:21.908514 12119 leader_election.cc:302] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 133012dfe80842df9b8c9864af5c1828, 7216c0f10096425f9ef797f9f9428c4f; no voters:
I20260620 14:53:21.908687 12384 raft_consensus.cc:2781] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:21.908919 12384 raft_consensus.cc:686] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 1 LEADER]: Becoming Leader. State: Replica: 133012dfe80842df9b8c9864af5c1828, State: Running, Role: LEADER
I20260620 14:53:21.909263 12384 consensus_queue.cc:237] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7216c0f10096425f9ef797f9f9428c4f" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43851 } } peers { permanent_uuid: "133012dfe80842df9b8c9864af5c1828" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36405 } } peers { permanent_uuid: "a5b0aba16a784ec3b37eff0bace9c750" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33759 } }
I20260620 14:53:21.912405 11909 catalog_manager.cc:5495] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 reported cstate change: term changed from 0 to 1, leader changed from <none> to 133012dfe80842df9b8c9864af5c1828 (127.7.24.194). New cstate: current_term: 1 leader_uuid: "133012dfe80842df9b8c9864af5c1828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7216c0f10096425f9ef797f9f9428c4f" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "133012dfe80842df9b8c9864af5c1828" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5b0aba16a784ec3b37eff0bace9c750" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33759 } health_report { overall_health: UNKNOWN } } }
W20260620 14:53:21.914062 12099 tablet.cc:2369] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:53:21.926715 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.925302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36502 (local address 127.7.24.194:36405)
0620 14:53:21.925421 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:21.925424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:21.925438 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:21.925594 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:21.925597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:21.925657 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:21.925727 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:21.926328 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:21.926331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:21.926333 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:21.926344 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:21.926362 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:21.926369 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:21.926445 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:21.926564 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 14:53:21.926603 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 14:53:21.932303 12314 raft_consensus.cc:1261] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f [term 1 FOLLOWER]: Refusing update from remote peer 133012dfe80842df9b8c9864af5c1828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:21.932307 12052 raft_consensus.cc:1261] T 1831f80cc7a142469d693e2dd40e48b3 P a5b0aba16a784ec3b37eff0bace9c750 [term 1 FOLLOWER]: Refusing update from remote peer 133012dfe80842df9b8c9864af5c1828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:21.933014 12384 consensus_queue.cc:1035] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5b0aba16a784ec3b37eff0bace9c750" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:21.933207 12387 consensus_queue.cc:1035] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7216c0f10096425f9ef797f9f9428c4f" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 43851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:21.940302 12390 mvcc.cc:205] Tried to move back new op lower bound from 7298937659108188160 to 7298937659022827520. Current Snapshot: MvccSnapshot[applied={T|T < 7298937659108188160}]
I20260620 14:53:21.942821 12392 mvcc.cc:205] Tried to move back new op lower bound from 7298937659108188160 to 7298937659022827520. Current Snapshot: MvccSnapshot[applied={T|T < 7298937659108188160 or (T in {7298937659108188160})}]
I20260620 14:53:21.949898 11909 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:21.950066 11909 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:21.952431 11909 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbbbf7d5dbc9406c8cc9f93eb4501dcb: Sending DeleteTablet for 3 replicas of tablet 1831f80cc7a142469d693e2dd40e48b3
I20260620 14:53:21.953202 12294 tablet_service.cc:1514] Processing DeleteTablet for tablet 1831f80cc7a142469d693e2dd40e48b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:21 UTC) from {username='slave'} at 127.0.0.1:58168
I20260620 14:53:21.953222 12032 tablet_service.cc:1514] Processing DeleteTablet for tablet 1831f80cc7a142469d693e2dd40e48b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:21 UTC) from {username='slave'} at 127.0.0.1:34496
I20260620 14:53:21.953708 12400 tablet_replica.cc:330] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f: stopping tablet replica
I20260620 14:53:21.953688 12163 tablet_service.cc:1514] Processing DeleteTablet for tablet 1831f80cc7a142469d693e2dd40e48b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:21 UTC) from {username='slave'} at 127.0.0.1:36488
I20260620 14:53:21.954021 12400 raft_consensus.cc:2229] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:21.954187 12402 tablet_replica.cc:330] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: stopping tablet replica
I20260620 14:53:21.954334 12400 raft_consensus.cc:2258] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:21.954568 12402 raft_consensus.cc:2229] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:21.954850 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 11970
I20260620 14:53:21.955056 12402 raft_consensus.cc:2258] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:21.955874 12400 ts_tablet_manager.cc:1905] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:21.956606 12402 ts_tablet_manager.cc:1905] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:21.958740 12400 ts_tablet_manager.cc:1918] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:21.958833 12400 log.cc:1196] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/1831f80cc7a142469d693e2dd40e48b3
I20260620 14:53:21.959146 12400 ts_tablet_manager.cc:1939] T 1831f80cc7a142469d693e2dd40e48b3 P 7216c0f10096425f9ef797f9f9428c4f: Deleting consensus metadata
I20260620 14:53:21.959985 12402 ts_tablet_manager.cc:1918] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:21.960098 12402 log.cc:1196] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/1831f80cc7a142469d693e2dd40e48b3
I20260620 14:53:21.960369 12402 ts_tablet_manager.cc:1939] T 1831f80cc7a142469d693e2dd40e48b3 P 133012dfe80842df9b8c9864af5c1828: Deleting consensus metadata
I20260620 14:53:21.961470 11894 catalog_manager.cc:4841] TS 133012dfe80842df9b8c9864af5c1828 (127.7.24.194:36405): tablet 1831f80cc7a142469d693e2dd40e48b3 (table test-table [id=7ba3141bb1584918a19cd76800e0a15c]) successfully deleted
I20260620 14:53:21.963354 11897 catalog_manager.cc:4841] TS 7216c0f10096425f9ef797f9f9428c4f (127.7.24.195:43851): tablet 1831f80cc7a142469d693e2dd40e48b3 (table test-table [id=7ba3141bb1584918a19cd76800e0a15c]) successfully deleted
W20260620 14:53:21.964871 11894 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:33759 (error 108)
I20260620 14:53:21.964943 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 12101
I20260620 14:53:21.966416 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:21.966112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33759 (local address 127.0.0.1:34510)
0620 14:53:21.966279 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:21.966338 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:33759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260620 14:53:21.966670 11894 catalog_manager.cc:4584] TS a5b0aba16a784ec3b37eff0bace9c750 (127.7.24.193:33759): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1831f80cc7a142469d693e2dd40e48b3: Network error: Client connection negotiation failed: client connection to 127.7.24.193:33759: connect: Connection refused (error 111)
I20260620 14:53:21.971300 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 12232
I20260620 14:53:21.976562 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 11879
2026-06-20T14:53:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (997 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260620 14:53:21.997886 7267 mini_oidc.cc:152] Starting JWKS server
I20260620 14:53:21.999439 7267 webserver.cc:460] Webserver started at https://127.0.0.1:33723/ using document root <none> and password file <none>
I20260620 14:53:21.999564 7267 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 14:53:21.999785 7267 webserver.cc:460] Webserver started at http://127.0.0.1:34365/ using document root <none> and password file <none>
2026-06-20T14:53:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:22Z Disabled control of system clock
I20260620 14:53:22.008536 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:44349
--webserver_interface=127.7.24.254
--webserver_port=0
--builtin_ntp_servers=127.7.24.212:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:44349
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33723/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/testchainca.cert with env {}
W20260620 14:53:22.116257 12409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:22.116509 12409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:22.116556 12409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:22.120524 12409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:22.120577 12409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:22.120592 12409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:22.120611 12409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:22.120625 12409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 14:53:22.120641 12409 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 14:53:22.120654 12409 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33723/jwks.json
I20260620 14:53:22.125834 12409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:44349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:44349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33723/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:22.126777 12409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:22.127681 12409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:22.135794 12417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:22.135794 12415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:22.135784 12414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:22.135888 12409 server_base.cc:1034] running on GCE node
I20260620 14:53:22.136504 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:22.137513 12409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:22.138685 12409 hybrid_clock.cc:648] HybridClock initialized: now 1781967202138664 us; error 38 us; skew 500 ppm
I20260620 14:53:22.140381 12409 webserver.cc:460] Webserver started at http://127.7.24.254:41441/ using document root <none> and password file <none>
I20260620 14:53:22.140919 12409 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:22.141001 12409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:22.141206 12409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:22.143208 12409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "e66a8cddd3d14584b1c1caf831cc9b45"
format_stamp: "Formatted at 2026-06-20 14:53:22 on dist-test-slave-sms9"
server_key: "2916d4e1f37baee37b56a5e1d54135a4"
server_key_iv: "61a90a290336e2d512b0cb416a5bf341"
server_key_version: "encryptionkey@0"
I20260620 14:53:22.143738 12409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "e66a8cddd3d14584b1c1caf831cc9b45"
format_stamp: "Formatted at 2026-06-20 14:53:22 on dist-test-slave-sms9"
server_key: "2916d4e1f37baee37b56a5e1d54135a4"
server_key_iv: "61a90a290336e2d512b0cb416a5bf341"
server_key_version: "encryptionkey@0"
I20260620 14:53:22.147193 12409 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:22.149345 12423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:22.150544 12409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:22.150662 12409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "e66a8cddd3d14584b1c1caf831cc9b45"
format_stamp: "Formatted at 2026-06-20 14:53:22 on dist-test-slave-sms9"
server_key: "2916d4e1f37baee37b56a5e1d54135a4"
server_key_iv: "61a90a290336e2d512b0cb416a5bf341"
server_key_version: "encryptionkey@0"
I20260620 14:53:22.150769 12409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:22.166929 12409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:22.167586 12409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:22.167789 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:22.174475 12409 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:44349
I20260620 14:53:22.174527 12475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:44349 every 8 connection(s)
I20260620 14:53:22.175387 12409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:22.178428 12476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:22.183682 12476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45: Bootstrap starting.
I20260620 14:53:22.183892 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 12409
I20260620 14:53:22.184019 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:22.184290 7267 external_mini_cluster.cc:1428] Setting key 033cfecbd95184c9517c8fcbff6b1f8e
I20260620 14:53:22.186295 12476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:22.187162 12476 log.cc:824] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:22.189170 12476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45: No bootstrap required, opened a new log
I20260620 14:53:22.193367 12476 raft_consensus.cc:348] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a8cddd3d14584b1c1caf831cc9b45" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44349 } }
I20260620 14:53:22.193617 12476 raft_consensus.cc:374] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:22.193671 12476 raft_consensus.cc:729] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e66a8cddd3d14584b1c1caf831cc9b45, State: Initialized, Role: FOLLOWER
I20260620 14:53:22.194118 12476 consensus_queue.cc:260] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a8cddd3d14584b1c1caf831cc9b45" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44349 } }
I20260620 14:53:22.194281 12476 raft_consensus.cc:388] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:22.194293 12479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:22.185543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52828 (local address 127.7.24.254:44349)
0620 14:53:22.186184 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:22.186205 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:22.186250 (+ 45us) server_negotiation.cc:407] Connection header received
0620 14:53:22.187126 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:22.187148 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:22.187628 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:22.187956 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:22.188350 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.189701 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.190605 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.191309 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.191466 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:22.192226 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:22.192235 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:22.192252 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:22.192811 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:22.192876 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:22.192901 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:22.192993 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:22.193121 (+ 128us) server_negotiation.cc:299] Negotiation successful
0620 14:53:22.193376 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":104,"threads_started":1}
I20260620 14:53:22.194370 12476 raft_consensus.cc:482] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:22.194473 12476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:22.195325 12476 raft_consensus.cc:504] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a8cddd3d14584b1c1caf831cc9b45" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44349 } }
I20260620 14:53:22.195650 12476 leader_election.cc:302] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e66a8cddd3d14584b1c1caf831cc9b45; no voters:
I20260620 14:53:22.195951 12476 leader_election.cc:288] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:22.196066 12481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:22.196285 12481 raft_consensus.cc:686] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [term 1 LEADER]: Becoming Leader. State: Replica: e66a8cddd3d14584b1c1caf831cc9b45, State: Running, Role: LEADER
I20260620 14:53:22.196573 12481 consensus_queue.cc:237] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a8cddd3d14584b1c1caf831cc9b45" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44349 } }
I20260620 14:53:22.196919 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:53:22.197189 12476 sys_catalog.cc:564] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:22.198354 12483 sys_catalog.cc:455] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e66a8cddd3d14584b1c1caf831cc9b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a8cddd3d14584b1c1caf831cc9b45" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44349 } } }
I20260620 14:53:22.198499 12483 sys_catalog.cc:458] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:22.198741 12482 sys_catalog.cc:455] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e66a8cddd3d14584b1c1caf831cc9b45. Latest consensus state: current_term: 1 leader_uuid: "e66a8cddd3d14584b1c1caf831cc9b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a8cddd3d14584b1c1caf831cc9b45" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44349 } } }
I20260620 14:53:22.198846 12482 sys_catalog.cc:458] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45 [sys.catalog]: This master's current role is: LEADER
W20260620 14:53:22.202057 12496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:53:22.202123 12496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:53:22.202203 12490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:22.202944 12490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:22.208190 12490 catalog_manager.cc:1347] Generated new cluster ID: f77cc769857c46339a987bdb104dd214
I20260620 14:53:22.208282 12490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:22.215327 12490 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:22.216238 12490 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:22.227258 12490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e66a8cddd3d14584b1c1caf831cc9b45: Generated new TSK 0
I20260620 14:53:22.228004 12490 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 14:53:22.258124 12506 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260620 14:53:22.258242 12506 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 14:53:22.258873 12479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:22.241489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.7.24.254:44349)
0620 14:53:22.241626 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:22.241632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:22.241822 (+ 190us) server_negotiation.cc:407] Connection header received
0620 14:53:22.242007 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:22.242011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:22.242066 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:22.242184 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0620 14:53:22.242400 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.243262 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.244129 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.244481 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.244584 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:22.244932 (+ 348us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 14:53:22.258426 (+ 13494us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 14:53:22.258722 (+ 296us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52832: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260620 14:53:22.258956 12479 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52832: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260620 14:53:22.260165 12505 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:22.241391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:44349 (local address 127.0.0.1:52832)
0620 14:53:22.241743 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:22.241755 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:22.241862 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:22.242196 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:22.242198 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:22.242205 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0620 14:53:22.242302 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:22.242307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.243391 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.243394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:22.244026 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:22.244033 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.244622 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.244625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:22.244700 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:22.244744 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 14:53:22.258676 (+ 13932us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 14:53:22.258888 (+ 212us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.24.254:44349: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":113,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.24.254:44349: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260620 14:53:22.260985 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 12409
2026-06-20T14:53:22Z chronyd exiting
I20260620 14:53:22.298036 7267 test_util.cc:175] -----------------------------------------------
I20260620 14:53:22.298137 7267 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781967176660325-7267-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (301 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260620 14:53:22.299208 7267 mini_oidc.cc:152] Starting JWKS server
I20260620 14:53:22.300048 7267 webserver.cc:460] Webserver started at https://127.0.0.1:35353/ using document root <none> and password file <none>
I20260620 14:53:22.300169 7267 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 14:53:22.300426 7267 webserver.cc:460] Webserver started at http://127.0.0.1:39871/ using document root <none> and password file <none>
2026-06-20T14:53:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:22Z Disabled control of system clock
I20260620 14:53:22.309930 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:37675
--webserver_interface=127.7.24.254
--webserver_port=0
--builtin_ntp_servers=127.7.24.212:37409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:37675
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35353/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/testchainca.cert with env {}
W20260620 14:53:22.417083 12512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:22.417373 12512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:22.417444 12512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:22.421861 12512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:22.421918 12512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:22.421934 12512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:22.421950 12512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:22.421967 12512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 14:53:22.421983 12512 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 14:53:22.421995 12512 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35353/jwks.json
I20260620 14:53:22.427203 12512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:37675
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:37675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35353/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:22.428122 12512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:22.429045 12512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:53:22.437594 12512 server_base.cc:1034] running on GCE node
W20260620 14:53:22.437546 12520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:22.437546 12517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:22.437552 12518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:22.438381 12512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:22.439548 12512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:22.440694 12512 hybrid_clock.cc:648] HybridClock initialized: now 1781967202440676 us; error 36 us; skew 500 ppm
I20260620 14:53:22.442426 12512 webserver.cc:460] Webserver started at http://127.7.24.254:44101/ using document root <none> and password file <none>
I20260620 14:53:22.443024 12512 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:22.443084 12512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:22.443322 12512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:22.445384 12512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "c5d7ebfe0f874be1802a4f31d780ff9b"
format_stamp: "Formatted at 2026-06-20 14:53:22 on dist-test-slave-sms9"
server_key: "e23db5217fc792cb8cd7028667961a8c"
server_key_iv: "5d46897cc67d82393fcc777d7fa63205"
server_key_version: "encryptionkey@0"
I20260620 14:53:22.445907 12512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "c5d7ebfe0f874be1802a4f31d780ff9b"
format_stamp: "Formatted at 2026-06-20 14:53:22 on dist-test-slave-sms9"
server_key: "e23db5217fc792cb8cd7028667961a8c"
server_key_iv: "5d46897cc67d82393fcc777d7fa63205"
server_key_version: "encryptionkey@0"
I20260620 14:53:22.449352 12512 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 14:53:22.451665 12526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:22.452898 12512 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:22.453025 12512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "c5d7ebfe0f874be1802a4f31d780ff9b"
format_stamp: "Formatted at 2026-06-20 14:53:22 on dist-test-slave-sms9"
server_key: "e23db5217fc792cb8cd7028667961a8c"
server_key_iv: "5d46897cc67d82393fcc777d7fa63205"
server_key_version: "encryptionkey@0"
I20260620 14:53:22.453130 12512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:22.480355 12512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:22.481050 12512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:22.481252 12512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:22.487922 12512 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:37675
I20260620 14:53:22.487934 12578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:37675 every 8 connection(s)
I20260620 14:53:22.488848 12512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:22.491914 12579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:22.495443 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 12512
I20260620 14:53:22.495575 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:22.495828 7267 external_mini_cluster.cc:1428] Setting key c8179f0b55edb8e1a6fd28ac4dbc30a6
I20260620 14:53:22.497680 12579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b: Bootstrap starting.
I20260620 14:53:22.500272 12579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:22.501106 12579 log.cc:824] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:22.503125 12579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b: No bootstrap required, opened a new log
I20260620 14:53:22.506228 12582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:22.497151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.7.24.254:37675)
0620 14:53:22.497789 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:22.497811 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:22.497851 (+ 40us) server_negotiation.cc:407] Connection header received
0620 14:53:22.498683 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:22.498705 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:22.499201 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:22.499545 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:22.499992 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.501476 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.502413 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.503090 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.503269 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:22.504035 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:22.504041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:22.504055 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:22.504544 (+ 489us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:22.504607 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:22.504631 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:22.504723 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:22.504858 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 14:53:22.505126 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":89,"threads_started":1}
I20260620 14:53:22.506912 12579 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d7ebfe0f874be1802a4f31d780ff9b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37675 } }
I20260620 14:53:22.507158 12579 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:22.507864 12579 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d7ebfe0f874be1802a4f31d780ff9b, State: Initialized, Role: FOLLOWER
I20260620 14:53:22.508307 12579 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d7ebfe0f874be1802a4f31d780ff9b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37675 } }
I20260620 14:53:22.508486 12579 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:22.508554 12579 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:22.508651 12579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:22.508955 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:53:22.509594 12579 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d7ebfe0f874be1802a4f31d780ff9b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37675 } }
I20260620 14:53:22.509969 12579 leader_election.cc:302] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5d7ebfe0f874be1802a4f31d780ff9b; no voters:
I20260620 14:53:22.510294 12579 leader_election.cc:288] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:22.510507 12584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:22.510759 12584 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [term 1 LEADER]: Becoming Leader. State: Replica: c5d7ebfe0f874be1802a4f31d780ff9b, State: Running, Role: LEADER
I20260620 14:53:22.511047 12584 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d7ebfe0f874be1802a4f31d780ff9b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37675 } }
I20260620 14:53:22.511449 12579 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:22.512589 12586 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5d7ebfe0f874be1802a4f31d780ff9b. Latest consensus state: current_term: 1 leader_uuid: "c5d7ebfe0f874be1802a4f31d780ff9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d7ebfe0f874be1802a4f31d780ff9b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37675 } } }
I20260620 14:53:22.512728 12586 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:22.513036 12585 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5d7ebfe0f874be1802a4f31d780ff9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d7ebfe0f874be1802a4f31d780ff9b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37675 } } }
I20260620 14:53:22.513123 12585 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:22.513201 12593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:22.516374 12593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:22.521799 12593 catalog_manager.cc:1347] Generated new cluster ID: 69b1170d38dd4e58941142d2fc325bdc
I20260620 14:53:22.521862 12593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:22.532974 12593 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:22.534132 12593 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:22.541476 12593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5d7ebfe0f874be1802a4f31d780ff9b: Generated new TSK 0
I20260620 14:53:22.542207 12593 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 14:53:22.565999 12609 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260620 14:53:22.566087 12609 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 14:53:22.566814 12582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:22.551123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38290 (local address 127.7.24.254:37675)
0620 14:53:22.551252 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:22.551257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:22.551394 (+ 137us) server_negotiation.cc:407] Connection header received
0620 14:53:22.551514 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:22.551517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:22.551573 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:22.551685 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0620 14:53:22.551861 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.552656 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.553494 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.553846 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.553947 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:22.554180 (+ 233us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 14:53:22.566360 (+ 12180us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35353/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 14:53:22.566658 (+ 298us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38290: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35353/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":44}
W20260620 14:53:22.566834 12608 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:22.551008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:37675 (local address 127.0.0.1:38290)
0620 14:53:22.551318 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:22.551330 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:22.551415 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:22.551679 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:22.551681 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:22.551687 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0620 14:53:22.551775 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:22.551779 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.552781 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.552784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:22.553391 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:22.553398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.553946 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.553949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:22.554009 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:22.554029 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 14:53:22.566568 (+ 12539us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35353/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 14:53:22.566699 (+ 131us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.24.254:37675: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35353/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":102,"threads_started":1}
W20260620 14:53:22.566876 12582 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38290: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35353/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260620 14:53:22.567584 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 12512
2026-06-20T14:53:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (307 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260620 14:53:22.606752 7267 mini_oidc.cc:152] Starting JWKS server
I20260620 14:53:22.607597 7267 webserver.cc:460] Webserver started at https://127.0.0.1:33793/ using document root <none> and password file <none>
I20260620 14:53:22.607726 7267 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 14:53:22.608006 7267 webserver.cc:460] Webserver started at http://127.0.0.1:37225/ using document root <none> and password file <none>
2026-06-20T14:53:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:22Z Disabled control of system clock
I20260620 14:53:22.616539 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:32817
--webserver_interface=127.7.24.254
--webserver_port=0
--builtin_ntp_servers=127.7.24.212:46877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:32817
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33793/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 14:53:22.726910 12615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:22.727200 12615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:22.727238 12615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:22.731312 12615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:22.731372 12615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:22.731392 12615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:22.731410 12615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:22.731424 12615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 14:53:22.731441 12615 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 14:53:22.731454 12615 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33793/jwks.json
I20260620 14:53:22.736131 12615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:46877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:32817
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:32817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33793/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:22.737134 12615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:22.738437 12615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:53:22.746713 12615 server_base.cc:1034] running on GCE node
W20260620 14:53:22.746657 12621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:22.746620 12620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:22.746639 12623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:22.747361 12615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:22.748430 12615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:22.749607 12615 hybrid_clock.cc:648] HybridClock initialized: now 1781967202749595 us; error 23 us; skew 500 ppm
I20260620 14:53:22.751359 12615 webserver.cc:460] Webserver started at http://127.7.24.254:34321/ using document root <none> and password file <none>
I20260620 14:53:22.751925 12615 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:22.752012 12615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:22.752230 12615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:22.754281 12615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "ec8ce34740de4aba82d91316a3dd5c51"
format_stamp: "Formatted at 2026-06-20 14:53:22 on dist-test-slave-sms9"
server_key: "5292c8fec145bcbb1fd44f20500fad1b"
server_key_iv: "69ed79efdee35d58a176d22655d90949"
server_key_version: "encryptionkey@0"
I20260620 14:53:22.754904 12615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "ec8ce34740de4aba82d91316a3dd5c51"
format_stamp: "Formatted at 2026-06-20 14:53:22 on dist-test-slave-sms9"
server_key: "5292c8fec145bcbb1fd44f20500fad1b"
server_key_iv: "69ed79efdee35d58a176d22655d90949"
server_key_version: "encryptionkey@0"
I20260620 14:53:22.758396 12615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:22.760748 12629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:22.761937 12615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:22.762079 12615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "ec8ce34740de4aba82d91316a3dd5c51"
format_stamp: "Formatted at 2026-06-20 14:53:22 on dist-test-slave-sms9"
server_key: "5292c8fec145bcbb1fd44f20500fad1b"
server_key_iv: "69ed79efdee35d58a176d22655d90949"
server_key_version: "encryptionkey@0"
I20260620 14:53:22.762189 12615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:22.784015 12615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:22.784721 12615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:22.784910 12615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:22.791592 12681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:32817 every 8 connection(s)
I20260620 14:53:22.791594 12615 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:32817
I20260620 14:53:22.792598 12615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:22.795855 12682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:22.801373 12682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51: Bootstrap starting.
I20260620 14:53:22.802120 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 12615
I20260620 14:53:22.802234 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:22.802512 7267 external_mini_cluster.cc:1428] Setting key 78b8e2d4eb6f969135fe650a7a258731
I20260620 14:53:22.804378 12682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:22.805099 12682 log.cc:824] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:22.807168 12682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51: No bootstrap required, opened a new log
I20260620 14:53:22.811447 12682 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ce34740de4aba82d91316a3dd5c51" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32817 } }
I20260620 14:53:22.811714 12682 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:22.811780 12682 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8ce34740de4aba82d91316a3dd5c51, State: Initialized, Role: FOLLOWER
I20260620 14:53:22.812119 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:22.803847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.7.24.254:32817)
0620 14:53:22.804503 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:22.804521 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:22.804556 (+ 35us) server_negotiation.cc:407] Connection header received
0620 14:53:22.805213 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:22.805241 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:22.805712 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:22.806040 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:22.806379 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.807439 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.808728 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.809220 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.809359 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:22.810195 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:22.810200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:22.810214 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:22.810670 (+ 456us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:22.810726 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:22.810763 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:22.810841 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:22.810986 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 14:53:22.811269 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":134,"threads_started":1}
I20260620 14:53:22.812261 12682 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ce34740de4aba82d91316a3dd5c51" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32817 } }
I20260620 14:53:22.812412 12682 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:22.812481 12682 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:22.812574 12682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:22.813625 12682 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ce34740de4aba82d91316a3dd5c51" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32817 } }
I20260620 14:53:22.813954 12682 leader_election.cc:302] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec8ce34740de4aba82d91316a3dd5c51; no voters:
I20260620 14:53:22.814291 12682 leader_election.cc:288] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:22.814402 12687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:22.814649 12687 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [term 1 LEADER]: Becoming Leader. State: Replica: ec8ce34740de4aba82d91316a3dd5c51, State: Running, Role: LEADER
I20260620 14:53:22.814955 12687 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ce34740de4aba82d91316a3dd5c51" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32817 } }
I20260620 14:53:22.815084 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:53:22.815541 12682 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:22.816614 12688 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec8ce34740de4aba82d91316a3dd5c51. Latest consensus state: current_term: 1 leader_uuid: "ec8ce34740de4aba82d91316a3dd5c51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ce34740de4aba82d91316a3dd5c51" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32817 } } }
I20260620 14:53:22.816740 12688 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:22.817039 12689 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec8ce34740de4aba82d91316a3dd5c51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8ce34740de4aba82d91316a3dd5c51" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32817 } } }
I20260620 14:53:22.817137 12689 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:22.817435 12696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:22.820011 12696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:22.825205 12696 catalog_manager.cc:1347] Generated new cluster ID: 2d6c487bc76440ea9c50404d84982e71
I20260620 14:53:22.825277 12696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:22.834863 12696 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:22.836107 12696 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:22.843115 12696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec8ce34740de4aba82d91316a3dd5c51: Generated new TSK 0
I20260620 14:53:22.843919 12696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:53:22.862311 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:22.857273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35888 (local address 127.7.24.254:32817)
0620 14:53:22.857417 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:22.857422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:22.857602 (+ 180us) server_negotiation.cc:407] Connection header received
0620 14:53:22.857761 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:22.857763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:22.857811 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:22.857900 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0620 14:53:22.858152 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.858751 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.859949 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.860158 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.860231 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:22.860549 (+ 318us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 14:53:22.861916 (+ 1367us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 14:53:22.862145 (+ 229us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":50}
W20260620 14:53:22.862389 12685 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260620 14:53:22.864177 12711 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:22.857176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32817 (local address 127.0.0.1:35888)
0620 14:53:22.857500 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:22.857516 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:22.857639 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:22.857932 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:22.857935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:22.857943 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0620 14:53:22.858045 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:22.858051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.858866 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.858870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:22.859840 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:22.859850 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:22.860277 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:22.860281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:22.860368 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:22.860395 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 14:53:22.862090 (+ 1695us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 14:53:22.863993 (+ 1903us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.24.254:32817: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":1694,"thread_start_us":107,"threads_started":1}
I20260620 14:53:22.864874 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 12615
E20260620 14:53:22.872932 12712 webserver.cc:533] Webserver: sslize failed:
E20260620 14:53:22.873019 12712 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-20T14:53:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (297 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-20T14:53:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:22Z Disabled control of system clock
I20260620 14:53:22.912297 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:37939
--webserver_interface=127.7.24.254
--webserver_port=0
--builtin_ntp_servers=127.7.24.212:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:37939
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/testchainca.cert with env {}
W20260620 14:53:23.019367 12716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:23.019624 12716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:23.019663 12716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:23.023541 12716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:23.023597 12716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:23.023612 12716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:23.023628 12716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:23.023643 12716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 14:53:23.023658 12716 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 14:53:23.023672 12716 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260620 14:53:23.028856 12716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:37939
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:37939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:23.029860 12716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:23.030817 12716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:23.039011 12724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:23.039044 12722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:23.039254 12716 server_base.cc:1034] running on GCE node
W20260620 14:53:23.039070 12721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:23.039892 12716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:23.041023 12716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:23.042239 12716 hybrid_clock.cc:648] HybridClock initialized: now 1781967203042134 us; error 121 us; skew 500 ppm
I20260620 14:53:23.044270 12716 webserver.cc:460] Webserver started at http://127.7.24.254:40061/ using document root <none> and password file <none>
I20260620 14:53:23.044903 12716 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:23.044999 12716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:23.045239 12716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:23.047478 12716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "add063455713461a98cf573cdb5ce22a"
format_stamp: "Formatted at 2026-06-20 14:53:23 on dist-test-slave-sms9"
server_key: "6ff4e4a002bb7c443c9fa34fa1da8935"
server_key_iv: "b1ee150eb436d9e285a8ea3c5e247735"
server_key_version: "encryptionkey@0"
I20260620 14:53:23.048072 12716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "add063455713461a98cf573cdb5ce22a"
format_stamp: "Formatted at 2026-06-20 14:53:23 on dist-test-slave-sms9"
server_key: "6ff4e4a002bb7c443c9fa34fa1da8935"
server_key_iv: "b1ee150eb436d9e285a8ea3c5e247735"
server_key_version: "encryptionkey@0"
I20260620 14:53:23.051947 12716 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 14:53:23.054605 12730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:23.055846 12716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:23.056006 12716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "add063455713461a98cf573cdb5ce22a"
format_stamp: "Formatted at 2026-06-20 14:53:23 on dist-test-slave-sms9"
server_key: "6ff4e4a002bb7c443c9fa34fa1da8935"
server_key_iv: "b1ee150eb436d9e285a8ea3c5e247735"
server_key_version: "encryptionkey@0"
I20260620 14:53:23.056133 12716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:23.084146 12716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:23.084895 12716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:23.085098 12716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:23.091702 12716 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:37939
I20260620 14:53:23.091727 12782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:37939 every 8 connection(s)
I20260620 14:53:23.092645 12716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:23.095388 12783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:23.097911 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 12716
I20260620 14:53:23.098040 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:23.098309 7267 external_mini_cluster.cc:1428] Setting key 45dece8a2891566e16b589658bf0a31f
I20260620 14:53:23.101114 12783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a: Bootstrap starting.
I20260620 14:53:23.103241 12783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:23.103945 12783 log.cc:824] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:23.105794 12783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a: No bootstrap required, opened a new log
I20260620 14:53:23.107209 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:23.099642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41182 (local address 127.7.24.254:37939)
0620 14:53:23.100274 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:23.100291 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:23.100323 (+ 32us) server_negotiation.cc:407] Connection header received
0620 14:53:23.100898 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:23.100914 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:23.101331 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:23.101644 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:23.102096 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:23.103116 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:23.103997 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:23.104523 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:23.104681 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:23.105553 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:23.105558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:23.105569 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:23.105967 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:23.106020 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:23.106039 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:23.106139 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:23.106269 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 14:53:23.106559 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":129,"threads_started":1}
I20260620 14:53:23.109697 12783 raft_consensus.cc:348] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add063455713461a98cf573cdb5ce22a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37939 } }
I20260620 14:53:23.109966 12783 raft_consensus.cc:374] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:23.110028 12783 raft_consensus.cc:729] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add063455713461a98cf573cdb5ce22a, State: Initialized, Role: FOLLOWER
I20260620 14:53:23.110030 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:53:23.110451 12783 consensus_queue.cc:260] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add063455713461a98cf573cdb5ce22a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37939 } }
I20260620 14:53:23.110630 12783 raft_consensus.cc:388] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:23.110690 12783 raft_consensus.cc:482] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:23.110785 12783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:23.111902 12783 raft_consensus.cc:504] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add063455713461a98cf573cdb5ce22a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37939 } }
I20260620 14:53:23.112181 12783 leader_election.cc:302] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: add063455713461a98cf573cdb5ce22a; no voters:
I20260620 14:53:23.112424 12783 leader_election.cc:288] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:23.112540 12788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:23.112792 12788 raft_consensus.cc:686] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [term 1 LEADER]: Becoming Leader. State: Replica: add063455713461a98cf573cdb5ce22a, State: Running, Role: LEADER
I20260620 14:53:23.113093 12788 consensus_queue.cc:237] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add063455713461a98cf573cdb5ce22a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37939 } }
I20260620 14:53:23.113667 12783 sys_catalog.cc:564] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:23.114826 12789 sys_catalog.cc:455] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "add063455713461a98cf573cdb5ce22a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add063455713461a98cf573cdb5ce22a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37939 } } }
I20260620 14:53:23.114817 12790 sys_catalog.cc:455] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [sys.catalog]: SysCatalogTable state changed. Reason: New leader add063455713461a98cf573cdb5ce22a. Latest consensus state: current_term: 1 leader_uuid: "add063455713461a98cf573cdb5ce22a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "add063455713461a98cf573cdb5ce22a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 37939 } } }
I20260620 14:53:23.114957 12789 sys_catalog.cc:458] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:23.114957 12790 sys_catalog.cc:458] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:23.115345 12797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:23.118145 12797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:23.123360 12797 catalog_manager.cc:1347] Generated new cluster ID: 3430a8ef3e62416aa2262210ad039d41
I20260620 14:53:23.123431 12797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:23.143958 12797 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:23.144886 12797 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:23.153978 12797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P add063455713461a98cf573cdb5ce22a: Generated new TSK 0
I20260620 14:53:23.154726 12797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:53:23.217710 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:23.164249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41190 (local address 127.7.24.254:37939)
0620 14:53:23.164372 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:23.164378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:23.164544 (+ 166us) server_negotiation.cc:407] Connection header received
0620 14:53:23.164740 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:23.164743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:23.164791 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:23.164904 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0620 14:53:23.165134 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:23.165685 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:23.166600 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:23.166828 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:23.166931 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:23.167250 (+ 319us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 14:53:23.217232 (+ 49982us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 14:53:23.217551 (+ 319us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41190: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 14:53:23.217808 12786 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41190: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260620 14:53:23.217926 12812 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:23.164176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:37939 (local address 127.0.0.1:41190)
0620 14:53:23.164464 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:23.164476 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:23.164595 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:23.164901 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:23.164905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:23.164912 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0620 14:53:23.165024 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:23.165030 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:23.165804 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:23.165809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:23.166450 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:23.166505 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:23.166954 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:23.166958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:23.167056 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:23.167079 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 14:53:23.217465 (+ 50386us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 14:53:23.217717 (+ 252us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.24.254:37939: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":102,"threads_started":1}
I20260620 14:53:23.218340 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 12716
2026-06-20T14:53:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (333 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (124 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (118 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:23 dist-test-slave-sms9 krb5kdc[12824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:23 dist-test-slave-sms9 krb5kdc[12824](info): set up 2 sockets
Jun 20 14:53:23 dist-test-slave-sms9 krb5kdc[12824](info): commencing operation
krb5kdc: starting...
W20260620 14:53:24.494696 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:24 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:24.616894 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:45493
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:45493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:24.726133 12837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:24.726389 12837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:24.726429 12837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:24.730393 12837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:24.730472 12837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:24.730543 12837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:24.730563 12837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:24.730587 12837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:24.735838 12837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:45493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:45493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:24.736869 12837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:24.737810 12837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:24.746052 12842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:24.746064 12843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:24.746204 12837 server_base.cc:1034] running on GCE node
W20260620 14:53:24.746064 12845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:24.746853 12837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:24.747957 12837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:24.749115 12837 hybrid_clock.cc:648] HybridClock initialized: now 1781967204749100 us; error 27 us; skew 500 ppm
Jun 20 14:53:24 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967204, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:24.751875 12837 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:24.752750 12837 webserver.cc:460] Webserver started at http://127.7.24.254:42097/ using document root <none> and password file <none>
I20260620 14:53:24.753297 12837 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:24.753355 12837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:24.753515 12837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:24.755510 12837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "3165c672130b4830a197d29b7ba2fc60"
format_stamp: "Formatted at 2026-06-20 14:53:24 on dist-test-slave-sms9"
server_key: "56030dd462507667fa71572dfc82abc8"
server_key_iv: "5f54df2435643c90668e1ca4f32c0181"
server_key_version: "encryptionkey@0"
I20260620 14:53:24.756021 12837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "3165c672130b4830a197d29b7ba2fc60"
format_stamp: "Formatted at 2026-06-20 14:53:24 on dist-test-slave-sms9"
server_key: "56030dd462507667fa71572dfc82abc8"
server_key_iv: "5f54df2435643c90668e1ca4f32c0181"
server_key_version: "encryptionkey@0"
I20260620 14:53:24.759588 12837 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260620 14:53:24.761782 12852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:24.762959 12837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:24.763055 12837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "3165c672130b4830a197d29b7ba2fc60"
format_stamp: "Formatted at 2026-06-20 14:53:24 on dist-test-slave-sms9"
server_key: "56030dd462507667fa71572dfc82abc8"
server_key_iv: "5f54df2435643c90668e1ca4f32c0181"
server_key_version: "encryptionkey@0"
I20260620 14:53:24.763134 12837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:24.798926 12837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:24.802023 12837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:24.802196 12837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:24.808970 12837 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:45493
I20260620 14:53:24.809036 12904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:45493 every 8 connection(s)
I20260620 14:53:24.809971 12837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:24.812784 12905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:24.813529 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 12837
I20260620 14:53:24.813629 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:24.813928 7267 external_mini_cluster.cc:1428] Setting key 7c2927fe487a5c4dd05b7d07d6a881e2
I20260620 14:53:24.819900 12905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60: Bootstrap starting.
I20260620 14:53:24.822227 12905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60: Neither blocks nor log segments found. Creating new log.
Jun 20 14:53:24 dist-test-slave-sms9 krb5kdc[12824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:24.823056 12905 log.cc:824] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:24.825044 12905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60: No bootstrap required, opened a new log
I20260620 14:53:24.828068 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:24.815436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.7.24.254:45493)
0620 14:53:24.815906 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:24.815920 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:24.815961 (+ 41us) server_negotiation.cc:407] Connection header received
0620 14:53:24.816686 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:24.816706 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:24.817035 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:24.817399 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:24.818182 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:24.819327 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:24.820209 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:24.820684 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:24.820858 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:24.823396 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:24.823401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:24.823414 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:24.823439 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:24.825287 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:24.825719 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:24.825725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:24.825735 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:24.825843 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:24.826141 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:24.826146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:24.826148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:24.826612 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:24.826814 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:24.827072 (+ 258us) server_negotiation.cc:299] Negotiation successful
0620 14:53:24.827362 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":140,"threads_started":1}
I20260620 14:53:24.828843 12905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165c672130b4830a197d29b7ba2fc60" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 45493 } }
I20260620 14:53:24.829106 12905 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:24.829172 12905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3165c672130b4830a197d29b7ba2fc60, State: Initialized, Role: FOLLOWER
I20260620 14:53:24.829614 12905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165c672130b4830a197d29b7ba2fc60" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 45493 } }
I20260620 14:53:24.829759 12905 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:24.829818 12905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:24.829908 12905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:24.830892 12905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165c672130b4830a197d29b7ba2fc60" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 45493 } }
I20260620 14:53:24.831166 12905 leader_election.cc:302] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3165c672130b4830a197d29b7ba2fc60; no voters:
I20260620 14:53:24.831399 12905 leader_election.cc:288] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:24.831524 12910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:24.831794 12910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [term 1 LEADER]: Becoming Leader. State: Replica: 3165c672130b4830a197d29b7ba2fc60, State: Running, Role: LEADER
I20260620 14:53:24.832080 12910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165c672130b4830a197d29b7ba2fc60" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 45493 } }
I20260620 14:53:24.832543 12905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:24.833612 12911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3165c672130b4830a197d29b7ba2fc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165c672130b4830a197d29b7ba2fc60" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 45493 } } }
I20260620 14:53:24.833738 12911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:24.834049 12912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3165c672130b4830a197d29b7ba2fc60. Latest consensus state: current_term: 1 leader_uuid: "3165c672130b4830a197d29b7ba2fc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165c672130b4830a197d29b7ba2fc60" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 45493 } } }
I20260620 14:53:24.834136 12912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:24.834221 12919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:24.837245 12919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:24.842418 12919 catalog_manager.cc:1347] Generated new cluster ID: cdae9e38353f42e087e212896f2f747d
I20260620 14:53:24.842525 12919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:24.860203 12919 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:24.861276 12919 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:24.872933 12919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3165c672130b4830a197d29b7ba2fc60: Generated new TSK 0
I20260620 14:53:24.873690 12919 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:53:24.909464 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:45493
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:25.021477 12931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:25.021730 12931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:25.021797 12931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:25.026355 12931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:25.026409 12931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:25.026508 12931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:25.031649 12931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:45493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:25.032624 12931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:25.033771 12931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:25.043387 12937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:25.043371 12939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:25.043359 12936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:25.043720 12931 server_base.cc:1034] running on GCE node
I20260620 14:53:25.044126 12931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:25.044802 12931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:25.045985 12931 hybrid_clock.cc:648] HybridClock initialized: now 1781967205045971 us; error 31 us; skew 500 ppm
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:25.052929 12931 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:53:25.053840 12931 webserver.cc:460] Webserver started at http://127.7.24.193:41887/ using document root <none> and password file <none>
I20260620 14:53:25.054392 12931 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:25.054503 12931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:25.054814 12931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:25.056833 12931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "76d7f9858acf4e66acfc9ec361eba9cc"
format_stamp: "Formatted at 2026-06-20 14:53:25 on dist-test-slave-sms9"
server_key: "5e1ed1bf687320581ef32b47c96bd0bd"
server_key_iv: "541005ff142e43b1b0e23f238c66e4e2"
server_key_version: "encryptionkey@0"
I20260620 14:53:25.057379 12931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "76d7f9858acf4e66acfc9ec361eba9cc"
format_stamp: "Formatted at 2026-06-20 14:53:25 on dist-test-slave-sms9"
server_key: "5e1ed1bf687320581ef32b47c96bd0bd"
server_key_iv: "541005ff142e43b1b0e23f238c66e4e2"
server_key_version: "encryptionkey@0"
I20260620 14:53:25.061035 12931 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 14:53:25.063361 12946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:25.064532 12931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:25.064703 12931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "76d7f9858acf4e66acfc9ec361eba9cc"
format_stamp: "Formatted at 2026-06-20 14:53:25 on dist-test-slave-sms9"
server_key: "5e1ed1bf687320581ef32b47c96bd0bd"
server_key_iv: "541005ff142e43b1b0e23f238c66e4e2"
server_key_version: "encryptionkey@0"
I20260620 14:53:25.064818 12931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:25.083427 12931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:25.086647 12931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:25.086863 12931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:25.087447 12931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:25.088258 12931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:25.088325 12931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:25.088392 12931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:25.088438 12931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:25.097236 12931 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:39787
I20260620 14:53:25.097263 13059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:39787 every 8 connection(s)
I20260620 14:53:25.098114 12931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:25.105779 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 12931
I20260620 14:53:25.105918 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:25.106282 7267 external_mini_cluster.cc:1428] Setting key 7434fb9542590a7234d9016de341fa97
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:25.112346 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:25.099952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:35981 (local address 127.7.24.254:45493)
0620 14:53:25.100107 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:25.100111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:25.100908 (+ 797us) server_negotiation.cc:407] Connection header received
0620 14:53:25.102189 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:25.102192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:25.102237 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:25.102311 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:25.103513 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.104042 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.105116 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.105339 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.105416 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:25.108250 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:25.108253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:25.108255 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:25.108277 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:25.110021 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.110722 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.110725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.110726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.110801 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.111192 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.111194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.111195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.111359 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:25.111431 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:25.112102 (+ 671us) server_negotiation.cc:299] Negotiation successful
0620 14:53:25.112226 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 14:53:25.113539 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:25.100313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:45493 (local address 127.7.24.193:35981)
0620 14:53:25.100739 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:25.100806 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:25.102009 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:25.102526 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:25.102534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:25.102915 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:25.103376 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:25.103389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.104148 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.104151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:25.105015 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:25.105024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.105461 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.105464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:25.105677 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:25.106524 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:25.106547 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:25.108088 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:25.110167 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:25.110172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:25.110185 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:25.110623 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:25.110895 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:25.110901 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:25.110906 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:25.111095 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:25.111459 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:25.111468 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:25.111808 (+ 340us) client_negotiation.cc:769] Sending connection context
0620 14:53:25.112020 (+ 212us) client_negotiation.cc:240] Negotiation successful
0620 14:53:25.112392 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":105,"threads_started":1}
I20260620 14:53:25.114795 13060 heartbeater.cc:347] Connected to a master server at 127.7.24.254:45493
I20260620 14:53:25.115058 13060 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:25.115674 13060 heartbeater.cc:511] Master 127.7.24.254:45493 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:53:25.117165 12869 ts_manager.cc:194] Registered new tserver with Master: 76d7f9858acf4e66acfc9ec361eba9cc (127.7.24.193:39787)
I20260620 14:53:25.118316 12869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:35981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:53:25.133852 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:45493
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:25.245373 13065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:25.245645 13065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:25.245688 13065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:25.249814 13065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:25.249883 13065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:25.249980 13065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:25.255563 13065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:45493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:25.256613 13065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:25.257603 13065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:53:25.267052 13065 server_base.cc:1034] running on GCE node
W20260620 14:53:25.267062 13071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:25.267071 13070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:25.267241 13073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:25.267663 13065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:25.268339 13065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:25.269526 13065 hybrid_clock.cc:648] HybridClock initialized: now 1781967205269501 us; error 39 us; skew 500 ppm
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:25.272290 13065 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:53:25.273200 13065 webserver.cc:460] Webserver started at http://127.7.24.194:37425/ using document root <none> and password file <none>
I20260620 14:53:25.273720 13065 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:25.273774 13065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:25.273936 13065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:25.275980 13065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "469b5968871848aa87e2ecea7c65c478"
format_stamp: "Formatted at 2026-06-20 14:53:25 on dist-test-slave-sms9"
server_key: "7e757024fe0d47137b753145ebef852f"
server_key_iv: "c17b77c263011f7a7897bfe37d7d627b"
server_key_version: "encryptionkey@0"
I20260620 14:53:25.276500 13065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "469b5968871848aa87e2ecea7c65c478"
format_stamp: "Formatted at 2026-06-20 14:53:25 on dist-test-slave-sms9"
server_key: "7e757024fe0d47137b753145ebef852f"
server_key_iv: "c17b77c263011f7a7897bfe37d7d627b"
server_key_version: "encryptionkey@0"
I20260620 14:53:25.280225 13065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260620 14:53:25.283077 13080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:25.284298 13065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:25.284451 13065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "469b5968871848aa87e2ecea7c65c478"
format_stamp: "Formatted at 2026-06-20 14:53:25 on dist-test-slave-sms9"
server_key: "7e757024fe0d47137b753145ebef852f"
server_key_iv: "c17b77c263011f7a7897bfe37d7d627b"
server_key_version: "encryptionkey@0"
I20260620 14:53:25.284571 13065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:25.298404 13065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:25.301290 13065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:25.301491 13065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:25.302064 13065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:25.302934 13065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:25.303001 13065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:25.303067 13065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:25.303115 13065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:25.311553 13065 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:38605
I20260620 14:53:25.311575 13193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:38605 every 8 connection(s)
I20260620 14:53:25.312481 13065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:25.320379 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 13065
I20260620 14:53:25.320479 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:25.320818 7267 external_mini_cluster.cc:1428] Setting key 545f5a0ed4276d39515f1b6fc1c5af05
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:25.328639 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:25.314487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:53641 (local address 127.7.24.254:45493)
0620 14:53:25.314676 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:25.314681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:25.316717 (+ 2036us) server_negotiation.cc:407] Connection header received
0620 14:53:25.318005 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:25.318008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:25.318055 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:25.318129 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:25.319285 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.319839 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.321244 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.321532 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.321617 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:25.324485 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:25.324488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:25.324490 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:25.324512 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:25.326235 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.326787 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.326794 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.326799 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.326897 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.327245 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.327248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.327250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.327550 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:25.327662 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:25.328239 (+ 577us) server_negotiation.cc:299] Negotiation successful
0620 14:53:25.328393 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 14:53:25.329211 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:25.314952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:45493 (local address 127.7.24.194:53641)
0620 14:53:25.316576 (+ 1624us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:25.316618 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:25.317824 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:25.318257 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:25.318264 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:25.318688 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:25.319153 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:25.319165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.320015 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.320042 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:25.321135 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:25.321145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.321664 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.321667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:25.321859 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:25.322523 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:25.322540 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:25.324248 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:25.326347 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:25.326351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:25.326364 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:25.326671 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:25.326990 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:25.326992 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:25.326993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:25.327116 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:25.327684 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:25.327693 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:25.327970 (+ 277us) client_negotiation.cc:769] Sending connection context
0620 14:53:25.328131 (+ 161us) client_negotiation.cc:240] Negotiation successful
0620 14:53:25.328383 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1406,"thread_start_us":142,"threads_started":1}
I20260620 14:53:25.330554 13194 heartbeater.cc:347] Connected to a master server at 127.7.24.254:45493
I20260620 14:53:25.330793 13194 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:25.331291 13194 heartbeater.cc:511] Master 127.7.24.254:45493 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:53:25.332300 12869 ts_manager.cc:194] Registered new tserver with Master: 469b5968871848aa87e2ecea7c65c478 (127.7.24.194:38605)
I20260620 14:53:25.333002 12869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:53641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:53:25.349351 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:45493
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:25.457516 13199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:25.457770 13199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:25.457813 13199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:25.461900 13199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:25.461966 13199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:25.462062 13199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:25.467631 13199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:45493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:25.468709 13199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:25.469730 13199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:25.479202 13207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:25.479212 13204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:25.479189 13205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:25.479362 13199 server_base.cc:1034] running on GCE node
I20260620 14:53:25.479820 13199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:25.480382 13199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:25.481531 13199 hybrid_clock.cc:648] HybridClock initialized: now 1781967205481517 us; error 24 us; skew 500 ppm
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:25.484292 13199 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:53:25.485139 13199 webserver.cc:460] Webserver started at http://127.7.24.195:36849/ using document root <none> and password file <none>
I20260620 14:53:25.485637 13199 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:25.485693 13199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:25.485852 13199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:25.487841 13199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "d80027ab98fb4c65a40ebabb411baab8"
format_stamp: "Formatted at 2026-06-20 14:53:25 on dist-test-slave-sms9"
server_key: "b0840a5bf704561f1514c15a5a50811d"
server_key_iv: "7cf20ff05953033e167e953c905736d8"
server_key_version: "encryptionkey@0"
I20260620 14:53:25.488374 13199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "d80027ab98fb4c65a40ebabb411baab8"
format_stamp: "Formatted at 2026-06-20 14:53:25 on dist-test-slave-sms9"
server_key: "b0840a5bf704561f1514c15a5a50811d"
server_key_iv: "7cf20ff05953033e167e953c905736d8"
server_key_version: "encryptionkey@0"
I20260620 14:53:25.492113 13199 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:53:25.494648 13214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:25.496063 13199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:25.496196 13199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "d80027ab98fb4c65a40ebabb411baab8"
format_stamp: "Formatted at 2026-06-20 14:53:25 on dist-test-slave-sms9"
server_key: "b0840a5bf704561f1514c15a5a50811d"
server_key_iv: "7cf20ff05953033e167e953c905736d8"
server_key_version: "encryptionkey@0"
I20260620 14:53:25.496316 13199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:25.515401 13199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:25.518658 13199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:25.518850 13199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:25.519419 13199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:25.520200 13199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:25.520254 13199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:25.520331 13199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:25.520356 13199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:25.528932 13199 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:45631
I20260620 14:53:25.529040 13327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:45631 every 8 connection(s)
I20260620 14:53:25.529850 13199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:25.535422 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 13199
I20260620 14:53:25.535573 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:25.535881 7267 external_mini_cluster.cc:1428] Setting key 9aae2071dd2e7c353f3eeb70707aab37
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:25.544392 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:25.531767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:44797 (local address 127.7.24.254:45493)
0620 14:53:25.531960 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:25.531964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:25.532902 (+ 938us) server_negotiation.cc:407] Connection header received
0620 14:53:25.534518 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:25.534522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:25.534582 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:25.534665 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:25.536288 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.537041 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.538253 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.538565 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.538682 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:25.541065 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:25.541069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:25.541071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:25.541097 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:25.542373 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.542914 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.542918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.542919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.543003 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.543344 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.543349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.543351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.543532 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:25.543637 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:25.544110 (+ 473us) server_negotiation.cc:299] Negotiation successful
0620 14:53:25.544231 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260620 14:53:25.545192 13330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:25.532219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:45493 (local address 127.7.24.195:44797)
0620 14:53:25.532748 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:25.532792 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:25.534271 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:25.534897 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:25.534904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:25.535392 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:25.536130 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:25.536150 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.537155 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.537158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:25.538040 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:25.538049 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.538713 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.538717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:25.538893 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:25.539507 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:25.539525 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:25.540908 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:25.542520 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:25.542525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:25.542541 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:25.542806 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:25.543097 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:25.543100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:25.543102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:25.543251 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:25.543635 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:25.543642 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:25.543877 (+ 235us) client_negotiation.cc:769] Sending connection context
0620 14:53:25.544059 (+ 182us) client_negotiation.cc:240] Negotiation successful
0620 14:53:25.544386 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":113,"threads_started":1}
I20260620 14:53:25.546291 13328 heartbeater.cc:347] Connected to a master server at 127.7.24.254:45493
I20260620 14:53:25.546556 13328 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:25.547021 13328 heartbeater.cc:511] Master 127.7.24.254:45493 requested a full tablet report, sending...
I20260620 14:53:25.548318 12869 ts_manager.cc:194] Registered new tserver with Master: d80027ab98fb4c65a40ebabb411baab8 (127.7.24.195:45631)
I20260620 14:53:25.549119 12869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:44797
I20260620 14:53:25.549850 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/test-admin.keytab.
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:25.566798 7267 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260620 14:53:25.569589 13335 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:25.568289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38986 (local address 127.7.24.193:39787)
0620 14:53:25.568540 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:25.568554 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:25.568577 (+ 23us) server_negotiation.cc:407] Connection header received
0620 14:53:25.568633 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:25.568637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:25.568837 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:25.568958 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:25.569396 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38986: BlockingRecv error: recv got EOF from 127.0.0.1:38986 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
W20260620 14:53:25.569619 7267 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/krb5cc' of type FILE
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:25.570716 7267 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:25.571678 7267 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:53:25.581284 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:25.572642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39000 (local address 127.7.24.193:39787)
0620 14:53:25.572802 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:25.572807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:25.572822 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:53:25.572955 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:25.572962 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:25.573001 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:25.573078 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:25.573605 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.574365 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.575306 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.575681 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.575800 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:25.578038 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:25.578047 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:25.578060 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:25.578082 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:25.579782 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.580161 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.580166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.580167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.580259 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.580473 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.580475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.580476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.580711 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:25.580875 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:25.581018 (+ 143us) server_negotiation.cc:299] Negotiation successful
0620 14:53:25.581137 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260620 14:53:25.584908 13335 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:25.584100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39010 (local address 127.7.24.193:39787)
0620 14:53:25.584228 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:25.584231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:25.584247 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:53:25.584349 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:25.584351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:25.584384 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:25.584445 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:25.584826 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39010: BlockingRecv error: recv got EOF from 127.0.0.1:39010 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260620 14:53:25.585028 7267 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/krb5cc' of type FILE
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:25.586035 7267 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:25.586930 7267 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:53:25.594877 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:25.587722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39014 (local address 127.7.24.193:39787)
0620 14:53:25.587830 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:25.587834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:25.587965 (+ 131us) server_negotiation.cc:407] Connection header received
0620 14:53:25.588048 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:25.588050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:25.588082 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:25.588137 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:25.588581 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.589136 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.589906 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.590111 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.590199 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:25.592155 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:25.592158 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:25.592160 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:25.592185 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:25.593527 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.593880 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.593882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.593883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.593946 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.594150 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.594152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.594153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.594303 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:25.594393 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:25.594602 (+ 209us) server_negotiation.cc:299] Negotiation successful
0620 14:53:25.594706 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 14:53:25.597357 13335 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:25.596493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39030 (local address 127.7.24.193:39787)
0620 14:53:25.596603 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:25.596608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:25.596686 (+ 78us) server_negotiation.cc:407] Connection header received
0620 14:53:25.596788 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:25.596792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:25.596832 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:25.596908 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:25.597271 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39030: BlockingRecv error: recv got EOF from 127.0.0.1:39030 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 14:53:25.597543 7267 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781967176660325-7267-0/krb5kdc/krb5cc' of type FILE
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:25.598551 7267 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:25.599539 7267 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[12824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:53:25.607587 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:25.600244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39036 (local address 127.7.24.193:39787)
0620 14:53:25.600357 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:25.600361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:25.600473 (+ 112us) server_negotiation.cc:407] Connection header received
0620 14:53:25.600592 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:25.600599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:25.600633 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:25.600707 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:25.601172 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.601674 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.602421 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:25.602672 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:25.602818 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:25.604915 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:25.604918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:25.604921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:25.604940 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:25.606262 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.606640 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.606643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.606644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.606764 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:25.606961 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:25.606963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:25.606964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:25.607088 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:25.607178 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:25.607327 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 14:53:25.607403 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 14:53:25.608525 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 12931
I20260620 14:53:25.614219 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 13065
I20260620 14:53:25.619515 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 13199
I20260620 14:53:25.624823 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 12837
2026-06-20T14:53:25Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2164 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[13348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[13348](info): set up 2 sockets
Jun 20 14:53:25 dist-test-slave-sms9 krb5kdc[13348](info): commencing operation
krb5kdc: starting...
W20260620 14:53:26.718927 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:26 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:26Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.254.keytab.
Entry for principal oryx/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.254.keytab.
I20260620 14:53:26.841717 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40517
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.254.keytab
--principal=oryx/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:40517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:26.957041 13361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:26.957453 13361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:26.957513 13361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:26.961613 13361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:26.961684 13361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:26.961704 13361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:26.961721 13361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:26.961736 13361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:26.967144 13361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:40517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.254.keytab
--principal=oryx/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:26.968170 13361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:26.969122 13361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:26.977864 13366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:26.977909 13369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:26.977850 13367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:26.977960 13361 server_base.cc:1034] running on GCE node
I20260620 14:53:26.978741 13361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:26.979822 13361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:26.980988 13361 hybrid_clock.cc:648] HybridClock initialized: now 1781967206980967 us; error 35 us; skew 500 ppm
Jun 20 14:53:26 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967206, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:26.983904 13361 init.cc:377] Logged in from keytab as oryx/127.7.24.254@KRBTEST.COM (short username oryx)
I20260620 14:53:26.984849 13361 webserver.cc:460] Webserver started at http://127.7.24.254:44131/ using document root <none> and password file <none>
I20260620 14:53:26.985414 13361 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:26.985502 13361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:26.985733 13361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:26.987856 13361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "d1f6ef82883544e98ed11e2e3dd8f786"
format_stamp: "Formatted at 2026-06-20 14:53:26 on dist-test-slave-sms9"
server_key: "ca093d162debed165da5e450c4295bf2"
server_key_iv: "0d2761cdf194565e558d63f7dc27dc9f"
server_key_version: "encryptionkey@0"
I20260620 14:53:26.988420 13361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "d1f6ef82883544e98ed11e2e3dd8f786"
format_stamp: "Formatted at 2026-06-20 14:53:26 on dist-test-slave-sms9"
server_key: "ca093d162debed165da5e450c4295bf2"
server_key_iv: "0d2761cdf194565e558d63f7dc27dc9f"
server_key_version: "encryptionkey@0"
I20260620 14:53:26.992201 13361 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:53:26.994685 13376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:26.995884 13361 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:26.996021 13361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "d1f6ef82883544e98ed11e2e3dd8f786"
format_stamp: "Formatted at 2026-06-20 14:53:26 on dist-test-slave-sms9"
server_key: "ca093d162debed165da5e450c4295bf2"
server_key_iv: "0d2761cdf194565e558d63f7dc27dc9f"
server_key_version: "encryptionkey@0"
I20260620 14:53:26.996129 13361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:27.014410 13361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:27.017554 13361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:27.017772 13361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:27.024890 13361 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:40517
I20260620 14:53:27.025177 13428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:40517 every 8 connection(s)
I20260620 14:53:27.025835 13361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:27.028214 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 13361
I20260620 14:53:27.028353 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:27.028671 7267 external_mini_cluster.cc:1428] Setting key e023173c07c1c73c778fce7aee0371d8
I20260620 14:53:27.028926 13429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:27.035862 13429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786: Bootstrap starting.
Jun 20 14:53:27 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:27.038550 13429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:27.039516 13429 log.cc:824] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:27.041302 13429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786: No bootstrap required, opened a new log
I20260620 14:53:27.043663 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:27.029994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55090 (local address 127.7.24.254:40517)
0620 14:53:27.030380 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:27.030395 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:27.030428 (+ 33us) server_negotiation.cc:407] Connection header received
0620 14:53:27.031105 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:27.031120 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:27.031422 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:27.031684 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:27.032553 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.033494 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.034653 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.035113 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.035228 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:27.038021 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:27.038026 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:27.038038 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:27.038064 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:27.040949 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:27.041387 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:27.041394 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:27.041399 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:27.041514 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:27.041795 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:27.041799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:27.041801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:27.042181 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:27.042394 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:27.042660 (+ 266us) server_negotiation.cc:299] Negotiation successful
0620 14:53:27.042967 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":126,"threads_started":1}
I20260620 14:53:27.044829 13429 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } }
I20260620 14:53:27.045063 13429 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:27.045137 13429 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f6ef82883544e98ed11e2e3dd8f786, State: Initialized, Role: FOLLOWER
I20260620 14:53:27.045535 13429 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } }
I20260620 14:53:27.045680 13429 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:27.045749 13429 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:27.045857 13429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:27.046903 13429 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } }
I20260620 14:53:27.047260 13429 leader_election.cc:302] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1f6ef82883544e98ed11e2e3dd8f786; no voters:
I20260620 14:53:27.047585 13429 leader_election.cc:288] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:27.047660 13434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:27.047940 13434 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 1 LEADER]: Becoming Leader. State: Replica: d1f6ef82883544e98ed11e2e3dd8f786, State: Running, Role: LEADER
I20260620 14:53:27.048525 13434 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } }
I20260620 14:53:27.048926 13429 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:27.050098 13435 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } }
I20260620 14:53:27.050225 13435 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:27.050644 13443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:27.050597 13436 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1f6ef82883544e98ed11e2e3dd8f786. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } }
I20260620 14:53:27.050806 13436 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:27.053529 13443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:27.058338 13443 catalog_manager.cc:1347] Generated new cluster ID: eb31eea6bdc4446dbf29f409f6d52a5d
I20260620 14:53:27.058408 13443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:27.085296 13443 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:27.086229 13443 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:27.091548 13443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786: Generated new TSK 0
I20260620 14:53:27.092245 13443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab.
Entry for principal oryx/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab.
I20260620 14:53:27.118884 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40517
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab
--principal=oryx/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:27.232409 13455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:27.232697 13455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:27.232741 13455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:27.236929 13455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:27.236999 13455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:27.237061 13455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:27.242535 13455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab
--principal=oryx/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:27.243558 13455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:27.244611 13455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:27.254926 13460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:27.254926 13461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:27.254930 13463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:27.255285 13455 server_base.cc:1034] running on GCE node
I20260620 14:53:27.255749 13455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:27.256469 13455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:27.257655 13455 hybrid_clock.cc:648] HybridClock initialized: now 1781967207257629 us; error 39 us; skew 500 ppm
Jun 20 14:53:27 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967207, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:27.260556 13455 init.cc:377] Logged in from keytab as oryx/127.7.24.193@KRBTEST.COM (short username oryx)
I20260620 14:53:27.261487 13455 webserver.cc:460] Webserver started at http://127.7.24.193:35589/ using document root <none> and password file <none>
I20260620 14:53:27.262101 13455 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:27.262190 13455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:27.262426 13455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:27.264647 13455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "d134ac5d57ad4d7c82e1d5d47f188909"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "e2f57c946a424793d7dabf1fb323d9d2"
server_key_iv: "a7ff06b7178dde59f5238f934d16688f"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.265213 13455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "d134ac5d57ad4d7c82e1d5d47f188909"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "e2f57c946a424793d7dabf1fb323d9d2"
server_key_iv: "a7ff06b7178dde59f5238f934d16688f"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.269443 13455 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260620 14:53:27.272032 13470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:27.273263 13455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:27.273401 13455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "d134ac5d57ad4d7c82e1d5d47f188909"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "e2f57c946a424793d7dabf1fb323d9d2"
server_key_iv: "a7ff06b7178dde59f5238f934d16688f"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.273509 13455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:27.297632 13455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:27.300910 13455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:27.301116 13455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:27.301721 13455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:27.302603 13455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:27.302670 13455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:27.302742 13455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:27.302780 13455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:27.311517 13455 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:39179
I20260620 14:53:27.311539 13583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:39179 every 8 connection(s)
I20260620 14:53:27.312382 13455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:27.315367 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 13455
I20260620 14:53:27.315507 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:27.315838 7267 external_mini_cluster.cc:1428] Setting key c8df56be40686db9fdf095359909f3f8
Jun 20 14:53:27 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967207, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:53:27.327337 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:27.314136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:41421 (local address 127.7.24.254:40517)
0620 14:53:27.314283 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:27.314286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:27.315104 (+ 818us) server_negotiation.cc:407] Connection header received
0620 14:53:27.316542 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:27.316545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:27.316609 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:27.316708 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:27.317966 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.318549 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.319668 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.319902 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.320008 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:27.323156 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:27.323162 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:27.323165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:27.323195 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:27.324975 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:27.325642 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:27.325645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:27.325646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:27.325725 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:27.326138 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:27.326141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:27.326142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:27.326336 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:27.326420 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:27.327059 (+ 639us) server_negotiation.cc:299] Negotiation successful
0620 14:53:27.327198 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 14:53:27.328248 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:27.314519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.193:41421)
0620 14:53:27.314955 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:27.314996 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:27.316340 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:27.316878 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:27.316884 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:27.317349 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:27.317824 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:27.317837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.318679 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.318682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:27.319545 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:27.319555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.320047 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.320051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:27.320219 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:27.320845 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:27.320861 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:27.322845 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:27.325095 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:27.325102 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:27.325119 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:27.325526 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:27.325826 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:27.325831 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:27.325834 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:27.326036 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:27.326435 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:27.326440 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:27.326803 (+ 363us) client_negotiation.cc:769] Sending connection context
0620 14:53:27.326978 (+ 175us) client_negotiation.cc:240] Negotiation successful
0620 14:53:27.327261 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.194@KRBTEST.COM" created.
I20260620 14:53:27.329526 13584 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40517
I20260620 14:53:27.329783 13584 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:27.330303 13584 heartbeater.cc:511] Master 127.7.24.254:40517 requested a full tablet report, sending...
I20260620 14:53:27.331939 13393 ts_manager.cc:194] Registered new tserver with Master: d134ac5d57ad4d7c82e1d5d47f188909 (127.7.24.193:39179)
I20260620 14:53:27.333155 13393 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:41421
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab.
Entry for principal oryx/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab.
I20260620 14:53:27.344934 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40517
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab
--principal=oryx/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:27.455329 13589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:27.455637 13589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:27.455709 13589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:27.459842 13589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:27.459930 13589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:27.460019 13589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:27.465570 13589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab
--principal=oryx/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:27.466727 13589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:27.467747 13589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:27.477216 13597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:27.477216 13595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:27.477377 13589 server_base.cc:1034] running on GCE node
W20260620 14:53:27.477226 13594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:27.477898 13589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:27.478633 13589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:27.479791 13589 hybrid_clock.cc:648] HybridClock initialized: now 1781967207479775 us; error 28 us; skew 500 ppm
Jun 20 14:53:27 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967207, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:27.482538 13589 init.cc:377] Logged in from keytab as oryx/127.7.24.194@KRBTEST.COM (short username oryx)
I20260620 14:53:27.483382 13589 webserver.cc:460] Webserver started at http://127.7.24.194:34907/ using document root <none> and password file <none>
I20260620 14:53:27.483930 13589 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:27.484015 13589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:27.484232 13589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:27.486222 13589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "a0bb85f6fa2348cda24dda06e035bf71"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "c640bc9c9743b5a0618102e6e6a87067"
server_key_iv: "77503592ac60de6efeb18d0c3945f1d9"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.486840 13589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "a0bb85f6fa2348cda24dda06e035bf71"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "c640bc9c9743b5a0618102e6e6a87067"
server_key_iv: "77503592ac60de6efeb18d0c3945f1d9"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.490626 13589 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 14:53:27.493003 13604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:27.494163 13589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:27.494318 13589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "a0bb85f6fa2348cda24dda06e035bf71"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "c640bc9c9743b5a0618102e6e6a87067"
server_key_iv: "77503592ac60de6efeb18d0c3945f1d9"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.494436 13589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:27.525110 13589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:27.528535 13589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:27.528740 13589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:27.529338 13589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:27.530164 13589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:27.530256 13589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:27.530332 13589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:27.530370 13589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:27.540307 13589 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:38787
I20260620 14:53:27.540328 13717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:38787 every 8 connection(s)
I20260620 14:53:27.541244 13589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:27.541497 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 13589
I20260620 14:53:27.541635 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:27.541965 7267 external_mini_cluster.cc:1428] Setting key ec6a96b6bd699f8a4bab28cccc825a4d
Jun 20 14:53:27 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967207, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.7.24.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.195@KRBTEST.COM" created.
I20260620 14:53:27.556650 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:27.543485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:47207 (local address 127.7.24.254:40517)
0620 14:53:27.543636 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:27.543641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:27.544498 (+ 857us) server_negotiation.cc:407] Connection header received
0620 14:53:27.546030 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:27.546036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:27.546097 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:27.546199 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:27.547418 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.547959 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.549746 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.550053 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.550186 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:27.553009 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:27.553013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:27.553016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:27.553046 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:27.554435 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:27.554961 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:27.554996 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:27.554998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:27.555117 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:27.555505 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:27.555508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:27.555509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:27.555744 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:27.555853 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:27.556341 (+ 488us) server_negotiation.cc:299] Negotiation successful
0620 14:53:27.556478 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 14:53:27.557641 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:27.543819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.194:47207)
0620 14:53:27.544326 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:27.544385 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:27.545835 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:27.546343 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:27.546350 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:27.546804 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:27.547291 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:27.547304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.548106 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.548112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:27.549610 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:27.549620 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.550226 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.550231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:27.550416 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:27.551154 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:27.551175 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:27.552840 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:27.554576 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:27.554582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:27.554599 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:27.554860 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:27.555218 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:27.555223 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:27.555225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:27.555361 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:27.555850 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:27.555858 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:27.556110 (+ 252us) client_negotiation.cc:769] Sending connection context
0620 14:53:27.556297 (+ 187us) client_negotiation.cc:240] Negotiation successful
0620 14:53:27.556624 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":109,"threads_started":1}
I20260620 14:53:27.558894 13718 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40517
I20260620 14:53:27.559124 13718 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:27.559617 13718 heartbeater.cc:511] Master 127.7.24.254:40517 requested a full tablet report, sending...
I20260620 14:53:27.560655 13393 ts_manager.cc:194] Registered new tserver with Master: a0bb85f6fa2348cda24dda06e035bf71 (127.7.24.194:38787)
I20260620 14:53:27.561225 13393 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:47207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab.
Entry for principal oryx/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab.
I20260620 14:53:27.573858 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40517
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab
--principal=oryx/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:27.684091 13723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:27.684365 13723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:27.684410 13723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:27.688372 13723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:27.688444 13723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:27.688505 13723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:27.693881 13723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab
--principal=oryx/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:27.695019 13723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:27.696046 13723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:27.706617 13731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:27.706657 13728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:27.706634 13729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:27.707166 13723 server_base.cc:1034] running on GCE node
I20260620 14:53:27.707598 13723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:27.708272 13723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:27.709482 13723 hybrid_clock.cc:648] HybridClock initialized: now 1781967207709440 us; error 57 us; skew 500 ppm
Jun 20 14:53:27 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967207, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:27.712407 13723 init.cc:377] Logged in from keytab as oryx/127.7.24.195@KRBTEST.COM (short username oryx)
I20260620 14:53:27.713352 13723 webserver.cc:460] Webserver started at http://127.7.24.195:44021/ using document root <none> and password file <none>
I20260620 14:53:27.713940 13723 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:27.714028 13723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:27.714254 13723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:27.716439 13723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "0827ede35a8e4df79127e7cebe970b5e"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "81f96706c419bddb2bd73645c5e066ca"
server_key_iv: "003af2eb44f096ad16e459e2d9f92790"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.717028 13723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "0827ede35a8e4df79127e7cebe970b5e"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "81f96706c419bddb2bd73645c5e066ca"
server_key_iv: "003af2eb44f096ad16e459e2d9f92790"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.720829 13723 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:53:27.723402 13738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:27.724684 13723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:27.724835 13723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "0827ede35a8e4df79127e7cebe970b5e"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "81f96706c419bddb2bd73645c5e066ca"
server_key_iv: "003af2eb44f096ad16e459e2d9f92790"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.724956 13723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:27.741412 13723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:27.744789 13723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:27.745007 13723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:27.745621 13723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:27.746623 13723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:27.746688 13723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:27.746755 13723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:27.746802 13723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:27.756713 13723 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:41681
I20260620 14:53:27.756747 13851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:41681 every 8 connection(s)
I20260620 14:53:27.757655 13723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:27.760496 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 13723
I20260620 14:53:27.760702 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:27.761027 7267 external_mini_cluster.cc:1428] Setting key abd34d2cee3397f101fd1c6fefca4ce0
Jun 20 14:53:27 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967207, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:27.772894 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:27.759732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:40933 (local address 127.7.24.254:40517)
0620 14:53:27.759887 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:27.759891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:27.760987 (+ 1096us) server_negotiation.cc:407] Connection header received
0620 14:53:27.762104 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:27.762107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:27.762163 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:27.762255 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:27.763691 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.764247 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.765412 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.765650 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.765744 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:27.768591 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:27.768595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:27.768597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:27.768624 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:27.770218 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:27.770897 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:27.770900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:27.770901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:27.770983 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:27.771501 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:27.771504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:27.771505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:27.771723 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:27.771952 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:27.772572 (+ 620us) server_negotiation.cc:299] Negotiation successful
0620 14:53:27.772752 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 14:53:27.773813 13854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:27.760061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.195:40933)
0620 14:53:27.760807 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:27.760861 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:27.761858 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:27.762458 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:27.762493 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:27.762972 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:27.763546 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:27.763559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.764375 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.764377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:27.765287 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:27.765297 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:27.765824 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:27.765827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:27.765982 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:27.766778 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:27.766795 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:27.768395 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:27.770371 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:27.770378 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:27.770397 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:27.770782 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:27.771146 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:27.771150 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:27.771152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:27.771319 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:27.771986 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:27.771992 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:27.772271 (+ 279us) client_negotiation.cc:769] Sending connection context
0620 14:53:27.772492 (+ 221us) client_negotiation.cc:240] Negotiation successful
0620 14:53:27.772799 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":139,"threads_started":1}
I20260620 14:53:27.775055 13852 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40517
I20260620 14:53:27.775333 13852 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:27.775916 13852 heartbeater.cc:511] Master 127.7.24.254:40517 requested a full tablet report, sending...
I20260620 14:53:27.776989 13393 ts_manager.cc:194] Registered new tserver with Master: 0827ede35a8e4df79127e7cebe970b5e (127.7.24.195:41681)
I20260620 14:53:27.777618 13393 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:40933
I20260620 14:53:27.787869 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.24.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.253.keytab.
Entry for principal oryx/127.7.24.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.253.keytab.
I20260620 14:53:27.819254 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.253:35457
--webserver_interface=127.7.24.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.253.keytab
--principal=oryx/127.7.24.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:40517,127.7.24.253:35457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:27.933375 13857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:27.933684 13857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:27.933727 13857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:27.937753 13857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:27.937816 13857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:27.937836 13857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:27.937853 13857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:27.937870 13857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:27.943086 13857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:40517,127.7.24.253:35457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.253.keytab
--principal=oryx/127.7.24.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.253:35457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.24.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
W20260620 14:53:27.943272 13857 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.24.254:40517,127.7.24.253:35457'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260620 14:53:27.944111 13857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:27.945135 13857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:27.953534 13863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:27.953516 13865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:27.953516 13862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:27.953843 13857 server_base.cc:1034] running on GCE node
I20260620 14:53:27.954335 13857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:27.955431 13857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:27.956579 13857 hybrid_clock.cc:648] HybridClock initialized: now 1781967207956557 us; error 37 us; skew 500 ppm
Jun 20 14:53:27 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967207, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:27.959378 13857 init.cc:377] Logged in from keytab as oryx/127.7.24.253@KRBTEST.COM (short username oryx)
I20260620 14:53:27.960295 13857 webserver.cc:460] Webserver started at http://127.7.24.253:39287/ using document root <none> and password file <none>
I20260620 14:53:27.960846 13857 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:27.960932 13857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:27.961162 13857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:27.963320 13857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data/instance:
uuid: "8838366add4347deb4f7275bd4f67ce1"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "390fd13f654178ba94727e6ba58e787a"
server_key_iv: "741eb63f6d280519ab4b68ff163d973c"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.963867 13857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/wal/instance:
uuid: "8838366add4347deb4f7275bd4f67ce1"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "390fd13f654178ba94727e6ba58e787a"
server_key_iv: "741eb63f6d280519ab4b68ff163d973c"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.967810 13857 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 14:53:27.970182 13872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:27.971472 13857 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:27.971611 13857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/wal
uuid: "8838366add4347deb4f7275bd4f67ce1"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "390fd13f654178ba94727e6ba58e787a"
server_key_iv: "741eb63f6d280519ab4b68ff163d973c"
server_key_version: "encryptionkey@0"
I20260620 14:53:27.971725 13857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:28.003937 13857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:28.007229 13857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:28.007431 13857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:28.014528 13857 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.253:35457
I20260620 14:53:28.014853 13924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.253:35457 every 8 connection(s)
I20260620 14:53:28.015535 13857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
I20260620 14:53:28.015890 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 13857
I20260620 14:53:28.018767 13925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:28.023378 13925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.24.253@KRBTEST.COM
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967207, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.253@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:28.032320 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.017498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60408 (local address 127.7.24.253:35457)
0620 14:53:28.017904 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.017922 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.017965 (+ 43us) server_negotiation.cc:407] Connection header received
0620 14:53:28.018853 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.018876 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.019285 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.019655 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.020342 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.021828 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.022741 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.023416 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.023581 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.026584 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.026591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.026602 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.026636 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.028591 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.029074 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.029080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.029085 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.029245 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.029562 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.029569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.029573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.030147 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.030372 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.031198 (+ 826us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.031415 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":107,"threads_started":1}
I20260620 14:53:28.035594 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.025367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55092 (local address 127.7.24.254:40517)
0620 14:53:28.025502 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.025509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.025924 (+ 415us) server_negotiation.cc:407] Connection header received
0620 14:53:28.026119 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.026122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.026176 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.026219 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.027205 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.028009 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.029253 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.029612 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.029709 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.032207 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.032211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.032213 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.032240 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.033722 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.034176 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.034181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.034183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.034266 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.034608 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.034611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.034613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.034851 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.034914 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.035306 (+ 392us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.035421 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:53:28.035631 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.025343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55092)
0620 14:53:28.025803 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.025852 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.026009 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.026435 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.026440 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.026902 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.027088 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.027099 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.028132 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.028135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.029128 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.029140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.029720 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.029724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.029833 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.030351 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.030364 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.032045 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.033836 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.033846 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.033851 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.034070 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.034363 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.034366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.034368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.034500 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.034978 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.034983 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.035086 (+ 103us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.035230 (+ 144us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.035422 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":209,"threads_started":1}
I20260620 14:53:28.036615 13925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967207, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.253@KRBTEST.COM for oryx/127.7.24.253@KRBTEST.COM
I20260620 14:53:28.045035 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.037159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.7.24.253:35457)
0620 14:53:28.037271 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.037276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.037310 (+ 34us) server_negotiation.cc:407] Connection header received
0620 14:53:28.037403 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.037406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.037446 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.037534 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.037932 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.038763 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.039453 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.039816 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.039909 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.042283 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.042286 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.042288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.042307 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.043549 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.043940 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.043947 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.043948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.044018 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.044269 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.044273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.044275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.044432 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.044534 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.044755 (+ 221us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.044914 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":38}
I20260620 14:53:28.045035 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.037087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35457 (local address 127.0.0.1:60414)
0620 14:53:28.037255 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.037272 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.037334 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.037538 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.037540 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.037718 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.037813 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.037818 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.038865 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.038867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.039357 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.039365 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.039923 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.039926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.039992 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.040447 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.040458 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.042127 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.043652 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.043658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.043661 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.043850 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.044109 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.044112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.044114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.044186 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.044523 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.044525 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.044609 (+ 84us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.044712 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.044858 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260620 14:53:28.048110 13925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: Bootstrap starting.
I20260620 14:53:28.050663 13925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:28.051406 13925 log.cc:824] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:28.053195 13925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: No bootstrap required, opened a new log
I20260620 14:53:28.056241 13925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } }
I20260620 14:53:28.056471 13925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:28.056545 13925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8838366add4347deb4f7275bd4f67ce1, State: Initialized, Role: FOLLOWER
I20260620 14:53:28.057006 13925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } }
I20260620 14:53:28.057484 13931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } } }
I20260620 14:53:28.057631 13931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:53:28.058158 13925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: configured and running, proceeding with master startup.
W20260620 14:53:28.062821 13942 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:53:28.062918 13942 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:53:28.065373 13857 master_runner.cc:428] Detected that this master 8838366add4347deb4f7275bd4f67ce1 is joining an existing cluster
I20260620 14:53:28.065444 13857 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.24.253:35457
I20260620 14:53:28.076987 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.067652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.7.24.253:35457)
0620 14:53:28.067958 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.067961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.068051 (+ 90us) server_negotiation.cc:407] Connection header received
0620 14:53:28.068160 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.068164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.068217 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.068287 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.068827 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.069302 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.070508 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.070712 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.070818 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.072139 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.072142 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.072145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.072178 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.074965 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.075474 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.075477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.075492 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.075584 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.075897 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.075900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.075902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.076170 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.076270 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.076571 (+ 301us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.076742 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":256}
I20260620 14:53:28.077198 13948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.067656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35457 (local address 127.0.0.1:60416)
0620 14:53:28.067944 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.067960 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.068064 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.068325 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.068328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.068563 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.068713 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.068720 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.069402 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.069405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.070359 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.070370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.070858 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.070862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.070979 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.071508 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.071523 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.072002 (+ 479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.075058 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.075061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.075063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.075395 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.075679 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.075682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.075683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.075783 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.076310 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.076314 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.076397 (+ 83us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.076500 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.076672 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":112,"threads_started":1}
I20260620 14:53:28.080353 13949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.067984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55094)
0620 14:53:28.071629 (+ 3645us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.071643 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.071770 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.072039 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.072042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.072259 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.072753 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.072763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.073823 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.073828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.074813 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.074823 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.075575 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.075583 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.075681 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.076131 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.076144 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.076536 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.078909 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.078914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.078916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.079110 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.079389 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.079392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.079394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.079475 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.079843 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.079847 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.079941 (+ 94us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.080040 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.080175 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3576,"thread_start_us":112,"threads_started":1}
I20260620 14:53:28.080355 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.067564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55094 (local address 127.7.24.254:40517)
0620 14:53:28.067685 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.067691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.071740 (+ 4049us) server_negotiation.cc:407] Connection header received
0620 14:53:28.071894 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.071897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.071942 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.072026 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.072876 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.073687 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.074930 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.075256 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.075382 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.076683 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.076686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.076689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.076715 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.078796 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.079205 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.079210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.079212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.079295 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.079561 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.079565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.079567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.079744 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.079843 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.080108 (+ 265us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.080216 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967206, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.254@KRBTEST.COM for oryx/127.7.24.253@KRBTEST.COM
I20260620 14:53:28.094378 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.085154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60432 (local address 127.7.24.253:35457)
0620 14:53:28.085265 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.085271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.085669 (+ 398us) server_negotiation.cc:407] Connection header received
0620 14:53:28.085925 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.085928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.085982 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.086072 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.086857 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.087638 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.088490 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.088717 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.088831 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.091124 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.091128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.091131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.091152 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.092657 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.093116 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.093121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.093122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.093199 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.093480 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.093483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.093485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.093658 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.093759 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.094084 (+ 325us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.094199 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 14:53:28.094379 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.085139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35457 (local address 127.0.0.1:60432)
0620 14:53:28.085551 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.085603 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.085763 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.086114 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.086120 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.086595 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.086725 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.086733 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.087736 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.087740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.088370 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.088380 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.088832 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.088836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.088957 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.089477 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.089489 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.090987 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.092770 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.092781 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.092788 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.093018 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.093297 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.093301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.093303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.093389 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.093767 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.093777 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.093893 (+ 116us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.094034 (+ 141us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.094224 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for oryx/127.7.24.252@KRBTEST.COM; defaulting to no policy
I20260620 14:53:28.096139 13393 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.24.253:35457: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8838366add4347deb4f7275bd4f67ce1"
member_type: NON_VOTER
last_known_addr {
host: "127.7.24.253"
port: 35457
}
attrs {
promote: true
}
}
dest_uuid: "d1f6ef82883544e98ed11e2e3dd8f786"
cas_config_opid_index: -1
I20260620 14:53:28.096801 13393 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.252@KRBTEST.COM" created.
I20260620 14:53:28.098788 13953 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8838366add4347deb4f7275bd4f67ce1 (127.7.24.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } } }
I20260620 14:53:28.099726 13899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:28.100836 13953 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.24.253:35457
I20260620 14:53:28.101068 13954 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } } }
I20260620 14:53:28.101171 13954 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:28.101306 13899 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 FOLLOWER]: Refusing update from remote peer d1f6ef82883544e98ed11e2e3dd8f786: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260620 14:53:28.101898 13857 master.cc:537] Master@127.7.24.253:35457 shutting down...
I20260620 14:53:28.101923 13954 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 14:53:28.103904 13378 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260620 14:53:28.106696 13857 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:28.106974 13857 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:28.107066 13857 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: stopping tablet replica
W20260620 14:53:28.108884 13378 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 -> Peer 8838366add4347deb4f7275bd4f67ce1 (127.7.24.253:35457): Couldn't send request to peer 8838366add4347deb4f7275bd4f67ce1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.252.keytab.
Entry for principal oryx/127.7.24.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.252.keytab.
I20260620 14:53:28.117197 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.252:36333
--webserver_interface=127.7.24.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.252.keytab
--principal=oryx/127.7.24.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:40517,127.7.24.253:35457,127.7.24.252:36333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:53:28.122485 13857 master.cc:552] Master@127.7.24.253:35457 shutdown complete.
I20260620 14:53:28.122548 13857 master_runner.cc:305] Clearing existing system tablet
I20260620 14:53:28.122848 13857 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:28.122915 13857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:28.124495 13857 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 14:53:28.125526 13960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.125741 13857 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:28.125866 13857 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:28.125900 13857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/wal
uuid: "8838366add4347deb4f7275bd4f67ce1"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "390fd13f654178ba94727e6ba58e787a"
server_key_iv: "741eb63f6d280519ab4b68ff163d973c"
server_key_version: "encryptionkey@0"
I20260620 14:53:28.128070 13857 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:28.131245 13857 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 14:53:28.131336 13857 log.cc:1196] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260620 14:53:28.131592 13857 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: Deleting consensus metadata
I20260620 14:53:28.131760 13857 master_runner.cc:315] Copying system tablet from 127.7.24.254:40517
I20260620 14:53:28.132943 13857 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.24.254:40517
I20260620 14:53:28.142004 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.133644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55102 (local address 127.7.24.254:40517)
0620 14:53:28.133773 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.133778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.134024 (+ 246us) server_negotiation.cc:407] Connection header received
0620 14:53:28.134152 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.134156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.134212 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.134301 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.134826 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.135665 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.136817 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.137173 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.137282 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.138580 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.138585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.138588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.138616 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.140323 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.140780 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.140784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.140786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.140870 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.141142 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.141145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.141147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.141356 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.141460 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.141732 (+ 272us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.141854 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 14:53:28.142012 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.133554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55102)
0620 14:53:28.133912 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.133927 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.134048 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.134329 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.134332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.134600 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.134716 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.134723 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.135782 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.135786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.136692 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.136703 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.137297 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.137300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.137397 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.137974 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.137989 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.138410 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.140436 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.140440 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.140442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.140692 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.140963 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.140966 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.140968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.141055 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.141459 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.141463 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.141559 (+ 96us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.141667 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.141806 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":92,"threads_started":1}
I20260620 14:53:28.142912 13413 tablet_copy_service.cc:140] P d1f6ef82883544e98ed11e2e3dd8f786: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8838366add4347deb4f7275bd4f67ce1 ({username='oryx', principal='oryx/127.7.24.253@KRBTEST.COM'} at 127.0.0.1:55102)
I20260620 14:53:28.143085 13413 tablet_copy_service.cc:161] P d1f6ef82883544e98ed11e2e3dd8f786: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8838366add4347deb4f7275bd4f67ce1 at {username='oryx', principal='oryx/127.7.24.253@KRBTEST.COM'} at 127.0.0.1:55102: session id = 8838366add4347deb4f7275bd4f67ce1-00000000000000000000000000000000
I20260620 14:53:28.144747 13413 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 14:53:28.146306 13857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 14:53:28.148583 13857 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: tablet copy: Starting download of 0 data blocks...
I20260620 14:53:28.148789 13857 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: tablet copy: Starting download of 1 WAL segments...
I20260620 14:53:28.150426 13857 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 14:53:28.152245 13413 tablet_copy_service.cc:342] P d1f6ef82883544e98ed11e2e3dd8f786: Request end of tablet copy session 8838366add4347deb4f7275bd4f67ce1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.24.253@KRBTEST.COM'} at 127.0.0.1:55102
I20260620 14:53:28.152370 13413 tablet_copy_service.cc:433] P d1f6ef82883544e98ed11e2e3dd8f786: ending tablet copy session 8838366add4347deb4f7275bd4f67ce1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8838366add4347deb4f7275bd4f67ce1
I20260620 14:53:28.159708 13857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:28.161398 13967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:28.163415 13968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:28.163610 13970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:28.163820 13857 server_base.cc:1034] running on GCE node
I20260620 14:53:28.163982 13857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:28.164330 13857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:28.165464 13857 hybrid_clock.cc:648] HybridClock initialized: now 1781967208165445 us; error 38 us; skew 500 ppm
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:28.167243 13857 init.cc:377] Logged in from keytab as oryx/127.7.24.253@KRBTEST.COM (short username oryx)
I20260620 14:53:28.167716 13857 webserver.cc:460] Webserver started at http://127.7.24.253:46719/ using document root <none> and password file <none>
I20260620 14:53:28.167917 13857 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:28.167959 13857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:28.169458 13857 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 14:53:28.174654 13977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.174930 13857 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.001s
I20260620 14:53:28.175021 13857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/wal
uuid: "8838366add4347deb4f7275bd4f67ce1"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "390fd13f654178ba94727e6ba58e787a"
server_key_iv: "741eb63f6d280519ab4b68ff163d973c"
server_key_version: "encryptionkey@0"
I20260620 14:53:28.175109 13857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:28.217828 13857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:28.220034 13857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:28.224646 13857 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.253:35457
I20260620 14:53:28.224689 14029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.253:35457 every 8 connection(s)
I20260620 14:53:28.225240 13857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
I20260620 14:53:28.227195 14030 sys_catalog.cc:263] Verifying existing consensus state
I20260620 14:53:28.228688 14030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: Bootstrap starting.
I20260620 14:53:28.239677 14030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:53:28.240001 14030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: Bootstrap complete.
I20260620 14:53:28.240579 14030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } }
I20260620 14:53:28.240716 14030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8838366add4347deb4f7275bd4f67ce1, State: Initialized, Role: LEARNER
I20260620 14:53:28.240831 14030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } }
I20260620 14:53:28.241169 14033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } } }
I20260620 14:53:28.241291 14030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:28.241369 14033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: This master's current role is: LEARNER
I20260620 14:53:28.244118 14044 catalog_manager.cc:1259] Loaded cluster ID: eb31eea6bdc4446dbf29f409f6d52a5d
I20260620 14:53:28.244254 14044 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: loading cluster ID for follower catalog manager: success
I20260620 14:53:28.245893 14044 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: acquiring CA information for follower catalog manager: success
I20260620 14:53:28.247056 14044 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260620 14:53:28.249311 13958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:28.249624 13958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:28.249699 13958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:28.254084 13958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:28.254137 13958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:28.254158 13958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:28.254194 13958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:28.254225 13958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:28.259697 13958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:40517,127.7.24.253:35457,127.7.24.252:36333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.252.keytab
--principal=oryx/127.7.24.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.252:36333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.24.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:28.260726 13958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:28.261704 13958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:28.270066 14049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:28.270061 14052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:28.270045 14050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:28.270107 13958 server_base.cc:1034] running on GCE node
I20260620 14:53:28.270812 13958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:28.271881 13958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:28.273068 13958 hybrid_clock.cc:648] HybridClock initialized: now 1781967208273051 us; error 31 us; skew 500 ppm
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:28.275769 13958 init.cc:377] Logged in from keytab as oryx/127.7.24.252@KRBTEST.COM (short username oryx)
I20260620 14:53:28.276659 13958 webserver.cc:460] Webserver started at http://127.7.24.252:42561/ using document root <none> and password file <none>
I20260620 14:53:28.277164 13958 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:28.277225 13958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:28.277385 13958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:28.279459 13958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data/instance:
uuid: "66840f8c385e4468a18e96be8573302c"
format_stamp: "Formatted at 2026-06-20 14:53:28 on dist-test-slave-sms9"
server_key: "bf667118ba446410515533f9459848a5"
server_key_iv: "cf6d0e381dcc9fbb7cf1fce7663cd1fd"
server_key_version: "encryptionkey@0"
I20260620 14:53:28.279953 13958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/wal/instance:
uuid: "66840f8c385e4468a18e96be8573302c"
format_stamp: "Formatted at 2026-06-20 14:53:28 on dist-test-slave-sms9"
server_key: "bf667118ba446410515533f9459848a5"
server_key_iv: "cf6d0e381dcc9fbb7cf1fce7663cd1fd"
server_key_version: "encryptionkey@0"
I20260620 14:53:28.283453 13958 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:53:28.285686 14059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.287024 13958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:28.287123 13958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/wal
uuid: "66840f8c385e4468a18e96be8573302c"
format_stamp: "Formatted at 2026-06-20 14:53:28 on dist-test-slave-sms9"
server_key: "bf667118ba446410515533f9459848a5"
server_key_iv: "cf6d0e381dcc9fbb7cf1fce7663cd1fd"
server_key_version: "encryptionkey@0"
I20260620 14:53:28.287218 13958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:28.317973 13958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:28.320771 13958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:28.320921 13958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:28.328531 13958 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.252:36333
I20260620 14:53:28.328826 14111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.252:36333 every 8 connection(s)
I20260620 14:53:28.329605 13958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
I20260620 14:53:28.332347 14112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:28.333535 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 13958
I20260620 14:53:28.335588 13584 heartbeater.cc:503] Master 127.7.24.254:40517 was elected leader, sending a full tablet report...
I20260620 14:53:28.336514 14112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.24.252@KRBTEST.COM
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.252@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:28.347982 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.338586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55112 (local address 127.7.24.254:40517)
0620 14:53:28.338703 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.338707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.338949 (+ 242us) server_negotiation.cc:407] Connection header received
0620 14:53:28.339051 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.339054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.339106 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.339174 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.339878 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.340551 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.341678 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.341971 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.342078 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.344616 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.344619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.344622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.344658 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.346138 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.346641 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.346645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.346653 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.346737 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.347041 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.347044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.347046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.347225 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.347323 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.347683 (+ 360us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.347796 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:53:28.348882 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.338377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55112)
0620 14:53:28.338818 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.338864 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.338976 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.339233 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.339238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.339638 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.339762 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.339767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.340659 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.340662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.341561 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.341569 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.342121 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.342125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.342279 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.342862 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.342877 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.344478 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.346247 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.346257 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.346267 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.346543 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.346822 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.346826 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.346828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.346945 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.347330 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.347339 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.347469 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.347622 (+ 153us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.348017 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":25,"thread_start_us":185,"threads_started":1}
I20260620 14:53:28.348883 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.335009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52470 (local address 127.7.24.252:36333)
0620 14:53:28.335419 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.335439 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.335475 (+ 36us) server_negotiation.cc:407] Connection header received
0620 14:53:28.336276 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.336299 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.336712 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.337142 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.337972 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.339389 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.340407 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.341019 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.341181 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.344487 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.344499 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.344510 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.344533 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.346085 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.346544 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.346550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.346551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.346671 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.346958 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.346962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.346964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.347319 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.347541 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.347710 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.348211 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":271,"server-negotiator.queue_time_us":274,"thread_start_us":147,"threads_started":1}
I20260620 14:53:28.350116 14112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.252@KRBTEST.COM for oryx/127.7.24.253@KRBTEST.COM
I20260620 14:53:28.357898 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.350586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35457 (local address 127.0.0.1:60448)
0620 14:53:28.350708 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.350723 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.350839 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.351153 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.351156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.351316 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.351397 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.351402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.352284 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.352290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.352817 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.352824 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.353292 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.353295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.353389 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.353863 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.353876 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.355050 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.356747 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.356749 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.356751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.356921 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.357141 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.357143 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.357144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.357212 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.357511 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.357513 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.357582 (+ 69us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.357656 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.357775 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 14:53:28.357919 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.350668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60448 (local address 127.7.24.253:35457)
0620 14:53:28.350941 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.350945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.350958 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:53:28.351000 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.351002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.351058 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.351139 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.351505 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.352182 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.352929 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.353186 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.353294 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.355189 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.355192 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.355194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.355217 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.356652 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.356996 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.356998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.356999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.357071 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.357279 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.357281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.357283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.357441 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.357512 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.357708 (+ 196us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.357803 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":106,"threads_started":1}
I20260620 14:53:28.358610 14112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.252@KRBTEST.COM for oryx/127.7.24.252@KRBTEST.COM
I20260620 14:53:28.365718 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.359084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:36333 (local address 127.0.0.1:52474)
0620 14:53:28.359184 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.359197 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.359261 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.359429 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.359431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.359550 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.359612 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.359615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.360633 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.360635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.361112 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.361119 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.361511 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.361513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.361566 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.361993 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.362004 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.363141 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.364504 (+ 1363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.364506 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.364508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.364689 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.364906 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.364908 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.364909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.364976 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.365266 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.365268 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.365333 (+ 65us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.365409 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.365540 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 14:53:28.365823 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.359119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52474 (local address 127.7.24.252:36333)
0620 14:53:28.359219 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.359223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.359245 (+ 22us) server_negotiation.cc:407] Connection header received
0620 14:53:28.359326 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.359329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.359360 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.359417 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.359690 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.360510 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.361212 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.361427 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.361500 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.363274 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.363277 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.363279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.363299 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.364415 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.364767 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.364769 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.364770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.364835 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.365043 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.365045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.365046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.365187 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.365267 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.365493 (+ 226us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.365607 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 14:53:28.368666 14112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: Bootstrap starting.
I20260620 14:53:28.371053 14112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:28.371724 14112 log.cc:824] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:28.373557 14112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: No bootstrap required, opened a new log
I20260620 14:53:28.376773 14112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } }
I20260620 14:53:28.376971 14112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:28.377003 14112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66840f8c385e4468a18e96be8573302c, State: Initialized, Role: FOLLOWER
I20260620 14:53:28.377440 14112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } }
I20260620 14:53:28.377928 14118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } } }
I20260620 14:53:28.378062 14118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:53:28.378667 14112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: configured and running, proceeding with master startup.
W20260620 14:53:28.383196 14129 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:53:28.383291 14129 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:53:28.386783 13958 master_runner.cc:428] Detected that this master 66840f8c385e4468a18e96be8573302c is joining an existing cluster
I20260620 14:53:28.386849 13958 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.24.252:36333
I20260620 14:53:28.398794 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.389245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60458 (local address 127.7.24.253:35457)
0620 14:53:28.389387 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.389392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.390292 (+ 900us) server_negotiation.cc:407] Connection header received
0620 14:53:28.390401 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.390405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.390479 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.390551 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.391319 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.392132 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.394019 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.394354 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.394423 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.395786 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.395790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.395793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.395816 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.397198 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.397626 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.397630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.397632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.397708 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.397975 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.397978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.397980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.398142 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.398250 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.398547 (+ 297us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.398665 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 14:53:28.398806 14135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.389373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35457 (local address 127.0.0.1:60458)
0620 14:53:28.390196 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.390207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.390317 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.390792 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.390796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.391070 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.391195 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.391201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.392281 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.392283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.393913 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.393921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.394824 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.394827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.394892 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.395328 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.395339 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.395639 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.397304 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.397310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.397313 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.397526 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.397796 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.397800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.397802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.397883 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.398239 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.398242 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.398347 (+ 105us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.398453 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.398604 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260620 14:53:28.402077 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 13455
I20260620 14:53:28.408798 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:39179
--local_ip_for_outbound_sockets=127.7.24.193
--tserver_master_addrs=127.7.24.254:40517
--webserver_port=35589
--webserver_interface=127.7.24.193
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab
--principal=oryx/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.24.254:40517,127.7.24.253:35457
--tserver_master_addrs=127.7.24.254:40517,127.7.24.253:35457,127.7.24.252:36333 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:53:28.433876 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.389566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52484 (local address 127.7.24.252:36333)
0620 14:53:28.389716 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.389720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.389736 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:53:28.389848 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.389851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.389893 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.389941 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.390621 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.391410 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.392568 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.392941 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.393048 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.394326 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.394330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.394333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.394355 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.432029 (+ 37674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.432551 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.432556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.432559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.432655 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.432943 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.432945 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.432947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.433221 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.433294 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.433587 (+ 293us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.433709 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":47}
I20260620 14:53:28.434530 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.389213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:36333 (local address 127.0.0.1:52484)
0620 14:53:28.389626 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.389641 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.389742 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.390064 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.390067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.390322 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.390501 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.390510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.391534 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.391537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.392447 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.392457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.393062 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.393066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.393160 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.393705 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.393720 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.394178 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.432164 (+ 37986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.432167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.432169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.432413 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.432762 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.432764 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.432767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.432858 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.433352 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.433354 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.433439 (+ 85us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.434207 (+ 768us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.434344 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":120,"threads_started":1}
I20260620 14:53:28.436416 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.389851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55118)
0620 14:53:28.390585 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.390597 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.390792 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.391202 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.391205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.391394 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.391494 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.391501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.392510 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.392514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.393737 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.393746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.394414 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.394418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.394524 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.431140 (+ 36616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.431175 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.431833 (+ 658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.433963 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.433966 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.433968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.434195 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.434820 (+ 625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.434823 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.434825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.434912 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.435922 (+ 1010us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.435925 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.436011 (+ 86us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.436106 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.436249 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260620 14:53:28.436969 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.389852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55118 (local address 127.7.24.254:40517)
0620 14:53:28.389991 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.389995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.390658 (+ 663us) server_negotiation.cc:407] Connection header received
0620 14:53:28.390913 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.390917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.390976 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.391044 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.391622 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.392378 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.393860 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.394196 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.394264 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.432076 (+ 37812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.432080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.432083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.432115 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.433832 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.434499 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.434503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.434505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.434601 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.435561 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.435565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.435567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.435783 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.435877 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.436699 (+ 822us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.436819 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967206, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.254@KRBTEST.COM for oryx/127.7.24.252@KRBTEST.COM
I20260620 14:53:28.449669 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.440718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:36333 (local address 127.0.0.1:52500)
0620 14:53:28.440836 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.440850 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.440931 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.441262 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.441265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.441448 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.441546 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.441554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.442575 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.442578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.443429 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.443439 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.444070 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.444074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.444163 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.444630 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.444646 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.446132 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.448112 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.448115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.448118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.448368 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.448652 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.448655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.448657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.448735 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.449159 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.449162 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.449264 (+ 102us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.449362 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.449500 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 14:53:28.450125 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.440783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52500 (local address 127.7.24.252:36333)
0620 14:53:28.440888 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.440893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.440905 (+ 12us) server_negotiation.cc:407] Connection header received
0620 14:53:28.441032 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.441035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.441082 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.441147 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.441668 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.442424 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.443558 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.443880 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.443965 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.446312 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.446316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.446319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.446346 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.447988 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.448476 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.448479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.448481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.448557 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.448833 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.448836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.448838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.449033 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.449117 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.449875 (+ 758us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.449983 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 14:53:28.451018 13393 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.24.252:36333: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "66840f8c385e4468a18e96be8573302c"
member_type: NON_VOTER
last_known_addr {
host: "127.7.24.252"
port: 36333
}
attrs {
promote: true
}
}
dest_uuid: "d1f6ef82883544e98ed11e2e3dd8f786"
cas_config_opid_index: 5
I20260620 14:53:28.451627 13393 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } }
I20260620 14:53:28.452677 13953 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 66840f8c385e4468a18e96be8573302c (127.7.24.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:28.454258 13953 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.24.252:36333
I20260620 14:53:28.454448 13953 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:28.454630 13953 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:28.454767 13958 master.cc:537] Master@127.7.24.252:36333 shutting down...
W20260620 14:53:28.456748 13379 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 -> Peer 66840f8c385e4468a18e96be8573302c (127.7.24.252:36333): Couldn't send request to peer 66840f8c385e4468a18e96be8573302c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 14:53:28.460520 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.453013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60474 (local address 127.7.24.253:35457)
0620 14:53:28.454816 (+ 1803us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.454821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.454838 (+ 17us) server_negotiation.cc:407] Connection header received
0620 14:53:28.454888 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.454891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.454950 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.455046 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:28.456363 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.457260 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.458955 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.459592 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.459676 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.459820 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.460318 (+ 498us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.460397 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1716}
I20260620 14:53:28.460553 13958 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:28.460824 13958 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:28.460927 13958 tablet_replica.cc:330] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: stopping tablet replica
I20260620 14:53:28.461181 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.452782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35457 (local address 127.0.0.1:60474)
0620 14:53:28.453123 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.453137 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.453208 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.456069 (+ 2861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.456072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.456092 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:28.456192 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.456201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.457501 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.457505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.458812 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.458823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.459917 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.459920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.460009 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.460065 (+ 56us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.460922 (+ 857us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.461058 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275}
I20260620 14:53:28.461843 14004 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 LEARNER]: Refusing update from remote peer d1f6ef82883544e98ed11e2e3dd8f786: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 14:53:28.462323 13954 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260620 14:53:28.464295 14004 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 66840f8c385e4468a18e96be8573302c (127.7.24.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:28.466547 14033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1f6ef82883544e98ed11e2e3dd8f786. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:28.466675 14033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: This master's current role is: LEARNER
I20260620 14:53:28.466770 14033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: NON_VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: true } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:28.466845 14033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: This master's current role is: LEARNER
I20260620 14:53:28.468231 14139 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786: attempting to promote NON_VOTER 8838366add4347deb4f7275bd4f67ce1 to VOTER
I20260620 14:53:28.468735 14139 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } }
I20260620 14:53:28.470063 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.469778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:36333 (local address 127.0.0.1:52516)
0620 14:53:28.469906 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.469981 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:36333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 14:53:28.470417 14004 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 LEARNER]: Refusing update from remote peer d1f6ef82883544e98ed11e2e3dd8f786: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260620 14:53:28.470849 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.470600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:36333 (local address 127.0.0.1:52532)
0620 14:53:28.470742 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.470788 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:36333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260620 14:53:28.470971 13379 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 -> Peer 66840f8c385e4468a18e96be8573302c (127.7.24.252:36333): Couldn't send request to peer 66840f8c385e4468a18e96be8573302c. Status: Network error: Client connection negotiation failed: client connection to 127.7.24.252:36333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 14:53:28.471081 13953 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260620 14:53:28.472613 13953 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8838366add4347deb4f7275bd4f67ce1 (127.7.24.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:28.473093 14004 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8838366add4347deb4f7275bd4f67ce1 (127.7.24.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:28.474049 13953 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:28.474146 13953 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:28.474261 13953 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:28.474326 13953 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:28.475055 14141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:28.475179 14141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:53:28.480430 13958 master.cc:552] Master@127.7.24.252:36333 shutdown complete.
I20260620 14:53:28.480486 13958 master_runner.cc:305] Clearing existing system tablet
I20260620 14:53:28.480724 13958 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:28.480780 13958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:28.482075 13958 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 14:53:28.482996 14143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.483227 13958 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:28.483498 13958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 14:53:28.483578 13958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/wal
uuid: "66840f8c385e4468a18e96be8573302c"
format_stamp: "Formatted at 2026-06-20 14:53:28 on dist-test-slave-sms9"
server_key: "bf667118ba446410515533f9459848a5"
server_key_iv: "cf6d0e381dcc9fbb7cf1fce7663cd1fd"
server_key_version: "encryptionkey@0"
I20260620 14:53:28.485709 13958 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:28.488222 13958 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 14:53:28.488381 13958 log.cc:1196] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260620 14:53:28.488675 13958 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: Deleting consensus metadata
I20260620 14:53:28.488876 13958 master_runner.cc:315] Copying system tablet from 127.7.24.254:40517
I20260620 14:53:28.490023 13958 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: tablet copy: Beginning tablet copy session from remote peer at address 127.7.24.254:40517
I20260620 14:53:28.503983 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.490774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55132 (local address 127.7.24.254:40517)
0620 14:53:28.490899 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.490904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.494811 (+ 3907us) server_negotiation.cc:407] Connection header received
0620 14:53:28.494930 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.494934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.494994 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.495099 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.495706 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.496558 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.497779 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.498133 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.498400 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.499990 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.499994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.499997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.500025 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.501838 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.502404 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.502408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.502410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.502576 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.502920 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.502923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.502925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.503158 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.503276 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.503652 (+ 376us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.503802 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 14:53:28.504596 14149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.490680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55132)
0620 14:53:28.494670 (+ 3990us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.494688 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.494835 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.495130 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.495133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.495360 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.495493 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.495500 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.496694 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.496699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.497615 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.497627 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.498401 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.498404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.498532 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.499231 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.499250 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.499807 (+ 557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.501962 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.501967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.501970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.502278 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.502683 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.502687 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.502689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.502810 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.503265 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.503269 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.503401 (+ 132us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.504162 (+ 761us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.504353 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3903,"thread_start_us":88,"threads_started":1}
I20260620 14:53:28.505160 13413 tablet_copy_service.cc:140] P d1f6ef82883544e98ed11e2e3dd8f786: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 66840f8c385e4468a18e96be8573302c ({username='oryx', principal='oryx/127.7.24.252@KRBTEST.COM'} at 127.0.0.1:55132)
I20260620 14:53:28.505273 13413 tablet_copy_service.cc:161] P d1f6ef82883544e98ed11e2e3dd8f786: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 66840f8c385e4468a18e96be8573302c at {username='oryx', principal='oryx/127.7.24.252@KRBTEST.COM'} at 127.0.0.1:55132: session id = 66840f8c385e4468a18e96be8573302c-00000000000000000000000000000000
I20260620 14:53:28.506444 13413 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 14:53:28.507987 13958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 14:53:28.510699 13958 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: tablet copy: Starting download of 0 data blocks...
I20260620 14:53:28.510910 13958 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: tablet copy: Starting download of 1 WAL segments...
I20260620 14:53:28.512454 13958 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 14:53:28.514295 13413 tablet_copy_service.cc:342] P d1f6ef82883544e98ed11e2e3dd8f786: Request end of tablet copy session 66840f8c385e4468a18e96be8573302c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.24.252@KRBTEST.COM'} at 127.0.0.1:55132
I20260620 14:53:28.514382 13413 tablet_copy_service.cc:433] P d1f6ef82883544e98ed11e2e3dd8f786: ending tablet copy session 66840f8c385e4468a18e96be8573302c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 66840f8c385e4468a18e96be8573302c
I20260620 14:53:28.523289 13958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:28.525156 14153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:28.525152 14150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:28.525650 13958 server_base.cc:1034] running on GCE node
W20260620 14:53:28.526607 14151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:28.526893 13958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:28.527197 13958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:28.528366 13958 hybrid_clock.cc:648] HybridClock initialized: now 1781967208528320 us; error 61 us; skew 500 ppm
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:28.530040 13958 init.cc:377] Logged in from keytab as oryx/127.7.24.252@KRBTEST.COM (short username oryx)
I20260620 14:53:28.530555 13958 webserver.cc:460] Webserver started at http://127.7.24.252:41879/ using document root <none> and password file <none>
I20260620 14:53:28.530756 13958 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:28.530802 13958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:28.532196 13958 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 14:53:28.533262 14160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.533514 13958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 14:53:28.533588 13958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/wal
uuid: "66840f8c385e4468a18e96be8573302c"
format_stamp: "Formatted at 2026-06-20 14:53:28 on dist-test-slave-sms9"
server_key: "bf667118ba446410515533f9459848a5"
server_key_iv: "cf6d0e381dcc9fbb7cf1fce7663cd1fd"
server_key_version: "encryptionkey@0"
I20260620 14:53:28.533664 13958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260620 14:53:28.540946 14138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:28.541265 14138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:28.541333 14138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260620 14:53:28.544103 13958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:28.546254 13958 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260620 14:53:28.547174 14138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:28.547250 14138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:28.547353 14138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:28.551409 13958 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.252:36333
I20260620 14:53:28.551433 14215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.252:36333 every 8 connection(s)
I20260620 14:53:28.551980 13958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
I20260620 14:53:28.553630 14216 sys_catalog.cc:263] Verifying existing consensus state
I20260620 14:53:28.554690 14216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: Bootstrap starting.
I20260620 14:53:28.554898 14138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab
--principal=oryx/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:39179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=35589
--tserver_master_addrs=127.7.24.254:40517,127.7.24.253:35457,127.7.24.252:36333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:28.556202 14138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:28.557430 14138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:53:28.563194 13718 heartbeater.cc:503] Master 127.7.24.254:40517 was elected leader, sending a full tablet report...
I20260620 14:53:28.565663 14216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:53:28.566118 14216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: Bootstrap complete.
I20260620 14:53:28.566680 14216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } }
I20260620 14:53:28.566824 14216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 66840f8c385e4468a18e96be8573302c, State: Initialized, Role: LEARNER
I20260620 14:53:28.566932 14216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } }
I20260620 14:53:28.567247 14224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:28.567359 14216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:28.567384 14224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: This master's current role is: LEARNER
W20260620 14:53:28.569417 14220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:28.569419 14223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:28.569829 14221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:28.570490 14138 server_base.cc:1034] running on GCE node
I20260620 14:53:28.570865 14236 catalog_manager.cc:1259] Loaded cluster ID: eb31eea6bdc4446dbf29f409f6d52a5d
I20260620 14:53:28.570917 14236 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: loading cluster ID for follower catalog manager: success
I20260620 14:53:28.571130 14138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:28.571882 14138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:28.572661 14236 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: acquiring CA information for follower catalog manager: success
I20260620 14:53:28.573076 14138 hybrid_clock.cc:648] HybridClock initialized: now 1781967208573041 us; error 50 us; skew 500 ppm
I20260620 14:53:28.574029 14236 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:28.576063 14138 init.cc:377] Logged in from keytab as oryx/127.7.24.193@KRBTEST.COM (short username oryx)
I20260620 14:53:28.576960 14138 webserver.cc:460] Webserver started at http://127.7.24.193:35589/ using document root <none> and password file <none>
I20260620 14:53:28.577499 14138 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:28.577597 14138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:28.581338 14138 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260620 14:53:28.583390 14242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.584481 14138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260620 14:53:28.584607 14138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "d134ac5d57ad4d7c82e1d5d47f188909"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "e2f57c946a424793d7dabf1fb323d9d2"
server_key_iv: "a7ff06b7178dde59f5238f934d16688f"
server_key_version: "encryptionkey@0"
I20260620 14:53:28.585141 14138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:28.610759 14138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:28.613669 14138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:28.613878 14138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:28.614506 14138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:28.615361 14138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:28.615437 14138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.615507 14138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:28.615566 14138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.624478 14138 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:39179
I20260620 14:53:28.624506 14355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:39179 every 8 connection(s)
I20260620 14:53:28.625459 14138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:28.629088 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 14138
I20260620 14:53:28.629280 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 13589
I20260620 14:53:28.636535 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:38787
--local_ip_for_outbound_sockets=127.7.24.194
--tserver_master_addrs=127.7.24.254:40517
--webserver_port=34907
--webserver_interface=127.7.24.194
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab
--principal=oryx/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.24.254:40517,127.7.24.253:35457
--tserver_master_addrs=127.7.24.254:40517,127.7.24.253:35457,127.7.24.252:36333 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.252@KRBTEST.COM
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.253@KRBTEST.COM
I20260620 14:53:28.644284 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.627247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:42185 (local address 127.7.24.253:35457)
0620 14:53:28.627406 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.627412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.628450 (+ 1038us) server_negotiation.cc:407] Connection header received
0620 14:53:28.630750 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.630754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.630811 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.630864 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.632756 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.633571 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.635481 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.635841 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.635916 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.640765 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.640768 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.640771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.640803 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.642580 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.643020 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.643025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.643026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.643098 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.643395 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.643398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.643400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.643590 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.643670 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.643995 (+ 325us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.644116 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 14:53:28.644289 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.627241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:58685 (local address 127.7.24.252:36333)
0620 14:53:28.627575 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.627580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.628980 (+ 1400us) server_negotiation.cc:407] Connection header received
0620 14:53:28.630843 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.630848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.630900 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.630980 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.632764 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.633994 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.635507 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.635902 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.635993 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.639689 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.639695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.639702 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.639736 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.641712 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.642368 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.642375 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.642378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.642495 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.642966 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.642972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.642975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.643218 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.643313 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.643995 (+ 682us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.644118 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":75,"threads_started":1}
I20260620 14:53:28.645107 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.627774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:32869 (local address 127.7.24.254:40517)
0620 14:53:28.627904 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.627909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.628887 (+ 978us) server_negotiation.cc:407] Connection header received
0620 14:53:28.630596 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.630599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.630655 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.630713 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.632755 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.633592 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.635481 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.635841 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.635916 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.640160 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.640167 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.640173 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.640208 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.642801 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.643359 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.643365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.643368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.643462 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.643832 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.643837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.643840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.644089 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.644207 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.644756 (+ 549us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.644887 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 14:53:28.646025 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.627767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.193:32869)
0620 14:53:28.628759 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.628772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.630312 (+ 1540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.631393 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.631396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.631816 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.632597 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.632607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.633811 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.633813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.635343 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.635354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.636115 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.636119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.636457 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.637331 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.637342 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.639939 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.642937 (+ 2998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.642941 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.642944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.643231 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.643576 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.643582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.643585 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.643691 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.644258 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.644263 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.644371 (+ 108us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.644467 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.645473 (+ 1006us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"mutex_wait_us":1089,"spinlock_wait_cycles":62464,"thread_start_us":149,"threads_started":1}
I20260620 14:53:28.646021 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.627740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:36333 (local address 127.7.24.193:58685)
0620 14:53:28.628862 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.628878 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.630530 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.631178 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.631187 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.631825 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.632587 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.632608 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.634143 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.634149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.635368 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.635381 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.636189 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.636192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.636457 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.637476 (+ 1019us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.637496 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.639503 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.641856 (+ 2353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.641862 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.641877 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.642233 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.642655 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.642659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.642663 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.642845 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.643378 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.643385 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.643685 (+ 300us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.644432 (+ 747us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.644804 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"mutex_wait_us":296,"spinlock_wait_cycles":27776,"thread_start_us":111,"threads_started":1}
I20260620 14:53:28.646023 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.627740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35457 (local address 127.7.24.193:42185)
0620 14:53:28.628306 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.628357 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.630312 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.631178 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.631187 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.631816 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.632598 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.632607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.633752 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.633757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.635343 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.635354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.636115 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.636119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.636423 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.637285 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.637310 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.640639 (+ 3329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.642686 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.642688 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.642691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.642918 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.643205 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.643208 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.643209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.643294 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.643725 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.643727 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.643808 (+ 81us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.644890 (+ 1082us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.645002 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":401,"spinlock_wait_cycles":37632,"thread_start_us":189,"threads_started":1}
I20260620 14:53:28.647301 14356 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40517
I20260620 14:53:28.647323 14357 heartbeater.cc:347] Connected to a master server at 127.7.24.252:36333
I20260620 14:53:28.647323 14358 heartbeater.cc:347] Connected to a master server at 127.7.24.253:35457
I20260620 14:53:28.647544 14358 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:28.647545 14356 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:28.647544 14357 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:28.648211 14358 heartbeater.cc:511] Master 127.7.24.253:35457 requested a full tablet report, sending...
I20260620 14:53:28.648200 14357 heartbeater.cc:511] Master 127.7.24.252:36333 requested a full tablet report, sending...
I20260620 14:53:28.648200 14356 heartbeater.cc:511] Master 127.7.24.254:40517 requested a full tablet report, sending...
I20260620 14:53:28.649498 13393 ts_manager.cc:194] Re-registered known tserver with Master: d134ac5d57ad4d7c82e1d5d47f188909 (127.7.24.193:39179)
I20260620 14:53:28.649732 14180 ts_manager.cc:194] Registered new tserver with Master: d134ac5d57ad4d7c82e1d5d47f188909 (127.7.24.193:39179)
I20260620 14:53:28.649816 13994 ts_manager.cc:194] Registered new tserver with Master: d134ac5d57ad4d7c82e1d5d47f188909 (127.7.24.193:39179)
I20260620 14:53:28.650198 13393 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:32869
W20260620 14:53:28.753881 14364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:28.754142 14364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:28.754194 14364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:28.758180 14364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:28.758230 14364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:28.758312 14364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:28.763567 14364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab
--principal=oryx/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:38787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=34907
--tserver_master_addrs=127.7.24.254:40517,127.7.24.253:35457,127.7.24.252:36333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:28.764554 14364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:28.765548 14364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:28.775597 14370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:28.775605 14372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:28.775662 14364 server_base.cc:1034] running on GCE node
W20260620 14:53:28.775605 14369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:28.776283 14364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:28.777010 14364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:28.778225 14364 hybrid_clock.cc:648] HybridClock initialized: now 1781967208778207 us; error 35 us; skew 500 ppm
I20260620 14:53:28.779866 13852 heartbeater.cc:503] Master 127.7.24.254:40517 was elected leader, sending a full tablet report...
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:28.781742 14364 init.cc:377] Logged in from keytab as oryx/127.7.24.194@KRBTEST.COM (short username oryx)
I20260620 14:53:28.782737 14364 webserver.cc:460] Webserver started at http://127.7.24.194:34907/ using document root <none> and password file <none>
I20260620 14:53:28.783303 14364 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:28.783409 14364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:28.787261 14364 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:53:28.789410 14379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.790769 14364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:28.790906 14364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "a0bb85f6fa2348cda24dda06e035bf71"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "c640bc9c9743b5a0618102e6e6a87067"
server_key_iv: "77503592ac60de6efeb18d0c3945f1d9"
server_key_version: "encryptionkey@0"
I20260620 14:53:28.791487 14364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:28.804557 14364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:28.807983 14364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:28.808203 14364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:28.808825 14364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:28.809722 14364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:28.809791 14364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.809864 14364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:28.809903 14364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.819257 14364 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:38787
I20260620 14:53:28.819280 14492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:38787 every 8 connection(s)
I20260620 14:53:28.820411 14364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:28.823808 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 14364
I20260620 14:53:28.824014 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 13723
I20260620 14:53:28.833218 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:41681
--local_ip_for_outbound_sockets=127.7.24.195
--tserver_master_addrs=127.7.24.254:40517
--webserver_port=44021
--webserver_interface=127.7.24.195
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab
--principal=oryx/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.24.254:40517,127.7.24.253:35457
--tserver_master_addrs=127.7.24.254:40517,127.7.24.253:35457,127.7.24.252:36333 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.252@KRBTEST.COM
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.253@KRBTEST.COM
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:28.841799 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.822466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:33921 (local address 127.7.24.252:36333)
0620 14:53:28.822684 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.822689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.824079 (+ 1390us) server_negotiation.cc:407] Connection header received
0620 14:53:28.826223 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.826229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.826299 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.826357 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.827838 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.828775 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.830878 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.831301 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.831400 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.836669 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.836675 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.836679 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.836717 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.839100 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.839807 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.839811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.839813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.839907 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.840349 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.840355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.840359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.840611 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.840701 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.841444 (+ 743us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.841597 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 14:53:28.841843 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.822275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:45307 (local address 127.7.24.253:35457)
0620 14:53:28.822420 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.822425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.823591 (+ 1166us) server_negotiation.cc:407] Connection header received
0620 14:53:28.825521 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.825525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.825587 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.825643 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.827831 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.828659 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.830744 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.831128 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.831225 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.837659 (+ 6434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.837666 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.837672 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.837710 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.839573 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.840152 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.840159 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.840161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.840255 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.840604 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.840609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.840612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.840877 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.840971 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.841465 (+ 494us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.841611 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 14:53:28.842854 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.822563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:49671 (local address 127.7.24.254:40517)
0620 14:53:28.822683 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:28.822687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:28.824768 (+ 2081us) server_negotiation.cc:407] Connection header received
0620 14:53:28.825522 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:28.825525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:28.825589 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:28.825643 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:28.827831 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.828781 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.833472 (+ 4691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.833854 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.833970 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.837894 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:28.837899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:28.837904 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:28.837938 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:28.840317 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.841004 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.841008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.841009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.841142 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:28.841619 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:28.841622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:28.841624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:28.841915 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:28.842171 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:28.842601 (+ 430us) server_negotiation.cc:299] Negotiation successful
0620 14:53:28.842713 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:53:28.843689 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.822893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.194:49671)
0620 14:53:28.824677 (+ 1784us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.824692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.825278 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.826081 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.826091 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.826797 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.827656 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.827675 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.828910 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.828912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.830677 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.833250 (+ 2573us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.834299 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.834303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.834407 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.835054 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.835072 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.837708 (+ 2636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.840484 (+ 2776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.840487 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.840489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.840854 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.841332 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.841365 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.841371 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.841487 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.842286 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.842289 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.842373 (+ 84us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.843055 (+ 682us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.843292 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"mutex_wait_us":536,"spinlock_wait_cycles":11520,"thread_start_us":89,"threads_started":1}
I20260620 14:53:28.843712 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.822893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:36333 (local address 127.7.24.194:33921)
0620 14:53:28.823989 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.824003 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.826073 (+ 2070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.826553 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.826558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.826837 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.827658 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.827678 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.828983 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.828989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.830710 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.830722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.831577 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.831583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.831767 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.833337 (+ 1570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.833481 (+ 144us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.836348 (+ 2867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.839244 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.839256 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.839280 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.839665 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.840030 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.840035 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.840037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.840233 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.840786 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.840795 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.841156 (+ 361us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.842495 (+ 1339us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.842646 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"mutex_wait_us":376,"spinlock_wait_cycles":2816,"thread_start_us":134,"threads_started":1}
I20260620 14:53:28.844064 14497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:28.822893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35457 (local address 127.7.24.194:45307)
0620 14:53:28.823421 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:28.823474 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:28.825268 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:28.826081 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:28.826091 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:28.826796 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:28.827656 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.827675 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.828842 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.828846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.830595 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:28.830608 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:28.831468 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:28.831473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:28.831740 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:28.833405 (+ 1665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:28.833420 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:28.837480 (+ 4060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:28.839717 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.839723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.839728 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.840009 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.840380 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:28.840385 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:28.840390 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:28.840487 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:28.841050 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:28.841056 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:28.841169 (+ 113us) client_negotiation.cc:769] Sending connection context
0620 14:53:28.842017 (+ 848us) client_negotiation.cc:240] Negotiation successful
0620 14:53:28.842444 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":328,"spinlock_wait_cycles":26880,"thread_start_us":149,"threads_started":1}
I20260620 14:53:28.845157 14495 heartbeater.cc:347] Connected to a master server at 127.7.24.253:35457
I20260620 14:53:28.845376 14493 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40517
I20260620 14:53:28.845417 14495 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:28.845438 14493 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:28.845992 14493 heartbeater.cc:511] Master 127.7.24.254:40517 requested a full tablet report, sending...
I20260620 14:53:28.845992 14495 heartbeater.cc:511] Master 127.7.24.253:35457 requested a full tablet report, sending...
I20260620 14:53:28.846081 14494 heartbeater.cc:347] Connected to a master server at 127.7.24.252:36333
I20260620 14:53:28.846318 14494 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:28.846503 14494 heartbeater.cc:511] Master 127.7.24.252:36333 requested a full tablet report, sending...
I20260620 14:53:28.847190 13393 ts_manager.cc:194] Re-registered known tserver with Master: a0bb85f6fa2348cda24dda06e035bf71 (127.7.24.194:38787)
I20260620 14:53:28.847241 14180 ts_manager.cc:194] Registered new tserver with Master: a0bb85f6fa2348cda24dda06e035bf71 (127.7.24.194:38787)
I20260620 14:53:28.847977 13393 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:49671
I20260620 14:53:28.848574 13994 ts_manager.cc:194] Registered new tserver with Master: a0bb85f6fa2348cda24dda06e035bf71 (127.7.24.194:38787)
W20260620 14:53:28.949600 14500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:28.949862 14500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:28.949927 14500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:28.953940 14500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:28.954025 14500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:28.954119 14500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:28.959237 14500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab
--principal=oryx/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:41681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=44021
--tserver_master_addrs=127.7.24.254:40517,127.7.24.253:35457,127.7.24.252:36333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:28.960253 14500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:28.961228 14500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:28.970957 14506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:28.970959 14505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:28.971025 14500 server_base.cc:1034] running on GCE node
W20260620 14:53:28.970973 14508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:28.971715 14500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:28.972383 14500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:28.973577 14500 hybrid_clock.cc:648] HybridClock initialized: now 1781967208973563 us; error 27 us; skew 500 ppm
Jun 20 14:53:28 dist-test-slave-sms9 krb5kdc[13348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:28.976624 14500 init.cc:377] Logged in from keytab as oryx/127.7.24.195@KRBTEST.COM (short username oryx)
I20260620 14:53:28.977526 14500 webserver.cc:460] Webserver started at http://127.7.24.195:44021/ using document root <none> and password file <none>
I20260620 14:53:28.978080 14500 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:28.978179 14500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:28.981961 14500 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:28.983985 14515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:28.985181 14500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:28.985298 14500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "0827ede35a8e4df79127e7cebe970b5e"
format_stamp: "Formatted at 2026-06-20 14:53:27 on dist-test-slave-sms9"
server_key: "81f96706c419bddb2bd73645c5e066ca"
server_key_iv: "003af2eb44f096ad16e459e2d9f92790"
server_key_version: "encryptionkey@0"
I20260620 14:53:28.985822 14500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:29.019690 14500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:29.023598 14500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:29.023810 14500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:29.024416 14500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:29.024935 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.020964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52536 (local address 127.7.24.252:36333)
0620 14:53:29.021089 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.021094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.021209 (+ 115us) server_negotiation.cc:407] Connection header received
0620 14:53:29.021372 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.021375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.021458 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.021630 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:29.021854 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.022425 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.023844 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.024372 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.024486 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.024666 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.024704 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.024800 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 14:53:29.024978 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.020844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:36333 (local address 127.0.0.1:52536)
0620 14:53:29.021109 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:29.021126 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:29.021267 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:29.021614 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:29.021618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:29.021639 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:29.021749 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.021756 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.022561 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.022564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.023716 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.023729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.024508 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.024512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.024602 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.024628 (+ 26us) client_negotiation.cc:769] Sending connection context
0620 14:53:29.024696 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 14:53:29.024777 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
I20260620 14:53:29.025486 14500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:29.025548 14500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:29.025597 14500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:29.025626 14500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:29.025739 14190 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260620 14:53:29.026166 14224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1f6ef82883544e98ed11e2e3dd8f786. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:29.026299 14224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: This master's current role is: LEARNER
I20260620 14:53:29.036362 14500 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:41681
I20260620 14:53:29.036386 14630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:41681 every 8 connection(s)
I20260620 14:53:29.037304 14500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:29.042241 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 14500
Jun 20 14:53:29 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.253@KRBTEST.COM
Jun 20 14:53:29 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
Jun 20 14:53:29 dist-test-slave-sms9 krb5kdc[13348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967208, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.252@KRBTEST.COM
I20260620 14:53:29.055043 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.044420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52540 (local address 127.7.24.252:36333)
0620 14:53:29.044647 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.044659 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.045027 (+ 368us) server_negotiation.cc:407] Connection header received
0620 14:53:29.045148 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.045151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.045197 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.045296 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:29.046258 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.047062 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.048184 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.048526 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.048605 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.050366 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:29.050370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:29.050372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:29.050397 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:29.052226 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.052734 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.052750 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.052751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.052837 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.053932 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.053936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.053938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.054184 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:29.054324 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.054663 (+ 339us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.054826 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260620 14:53:29.059911 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.039694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.195:60867)
0620 14:53:29.040875 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:29.040886 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:29.041234 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:29.041720 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:29.041726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:29.042494 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:29.043037 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.043054 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.044104 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.044107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.046997 (+ 2890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.047009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.047685 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.047688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.047777 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.048249 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:29.048263 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:29.052098 (+ 3835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:29.057769 (+ 5671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:29.057773 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:29.057775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:29.058080 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:29.058378 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:29.058384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:29.058389 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:29.058515 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:29.058999 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:29.059002 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:29.059093 (+ 91us) client_negotiation.cc:769] Sending connection context
0620 14:53:29.059181 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 14:53:29.059329 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"mutex_wait_us":1836,"spinlock_wait_cycles":11648,"thread_start_us":104,"threads_started":1}
I20260620 14:53:29.060041 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.049200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55134 (local address 127.7.24.254:40517)
0620 14:53:29.050878 (+ 1678us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.050882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.050894 (+ 12us) server_negotiation.cc:407] Connection header received
0620 14:53:29.050928 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.050931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.050981 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.051074 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:29.052014 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.052833 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.054060 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.054396 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.054541 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.056036 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:29.056040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:29.056044 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:29.056071 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:29.057965 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.058479 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.058482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.058484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.058563 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.058996 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.058999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.059001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.059189 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:29.059314 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.059761 (+ 447us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.059878 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1604,"thread_start_us":103,"threads_started":1}
I20260620 14:53:29.060624 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.040452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:49161 (local address 127.7.24.252:36333)
0620 14:53:29.040580 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.040584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.040730 (+ 146us) server_negotiation.cc:407] Connection header received
0620 14:53:29.041449 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.041452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.041508 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.041579 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:29.043173 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.043984 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.048026 (+ 4042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.048400 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.048481 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.053401 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:29.053404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:29.053407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:29.053431 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:29.055883 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.056792 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.056795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.056797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.056875 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.057320 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.057326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.057328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.057560 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:29.057666 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.060363 (+ 2697us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.060484 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 14:53:29.060223 14637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.039482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:36333 (local address 127.7.24.195:49161)
0620 14:53:29.040647 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:29.040660 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:29.041232 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:29.041738 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:29.041741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:29.042388 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:29.043035 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.043050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.044104 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.044107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.047891 (+ 3784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.047902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.048660 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.048664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.048780 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.049445 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:29.049458 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:29.053252 (+ 3794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:29.056286 (+ 3034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:29.056293 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:29.056313 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:29.056663 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:29.056986 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:29.056992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:29.056995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:29.057199 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:29.057696 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:29.057706 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:29.058062 (+ 356us) client_negotiation.cc:769] Sending connection context
0620 14:53:29.058312 (+ 250us) client_negotiation.cc:240] Negotiation successful
0620 14:53:29.058738 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1090,"mutex_wait_us":2739,"spinlock_wait_cycles":34688,"thread_start_us":146,"threads_started":1}
I20260620 14:53:29.061678 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.044728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.7.24.253:35457)
0620 14:53:29.045331 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.045335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.045605 (+ 270us) server_negotiation.cc:407] Connection header received
0620 14:53:29.045705 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.045707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.045755 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.045856 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:29.046646 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.048929 (+ 2283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.050197 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.050592 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.050699 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.052330 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:29.052333 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:29.052336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:29.052362 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:29.059459 (+ 7097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.059972 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.059976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.059977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.060071 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.060398 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.060401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.060403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.060611 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:29.060755 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.061376 (+ 621us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.061507 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":104,"threads_started":1}
I20260620 14:53:29.062278 14632 heartbeater.cc:347] Connected to a master server at 127.7.24.252:36333
I20260620 14:53:29.062551 14632 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:29.062680 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.039820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:60867 (local address 127.7.24.254:40517)
0620 14:53:29.039943 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.039947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.040955 (+ 1008us) server_negotiation.cc:407] Connection header received
0620 14:53:29.041429 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.041432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.041487 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.041553 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:29.043223 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.043926 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.047121 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.047470 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.047570 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.052454 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:29.052458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:29.052461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:29.052485 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:29.057668 (+ 5183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.058188 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.058192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.058193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.058271 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.058621 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.058624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.058625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.058852 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:29.058950 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.062393 (+ 3443us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.062540 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:53:29.063122 14632 heartbeater.cc:511] Master 127.7.24.252:36333 requested a full tablet report, sending...
I20260620 14:53:29.063221 14631 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40517
I20260620 14:53:29.063311 14631 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:29.063476 14631 heartbeater.cc:511] Master 127.7.24.254:40517 requested a full tablet report, sending...
I20260620 14:53:29.064073 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.039482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35457 (local address 127.7.24.195:43675)
0620 14:53:29.040174 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:29.040223 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:29.041232 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:29.041747 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:29.041750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:29.042380 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:29.043265 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.043272 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.044268 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.044274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.045552 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.045560 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.046163 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.046167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.046386 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.047073 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:29.047097 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:29.053252 (+ 6155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:29.057794 (+ 4542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:29.057797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:29.057799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:29.060332 (+ 2533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:29.063012 (+ 2680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:29.063015 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:29.063017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:29.063104 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:29.063555 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:29.063558 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:29.063644 (+ 86us) client_negotiation.cc:769] Sending connection context
0620 14:53:29.063757 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 14:53:29.063910 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":86,"thread_start_us":128,"threads_started":1}
I20260620 14:53:29.064119 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.039095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:43675 (local address 127.7.24.253:35457)
0620 14:53:29.040465 (+ 1370us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.040468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.040502 (+ 34us) server_negotiation.cc:407] Connection header received
0620 14:53:29.041462 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.041466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.041511 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.041578 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:29.043389 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.044130 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.045695 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.046019 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.046142 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.053401 (+ 7259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:29.053404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:29.053407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:29.053432 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:29.056902 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.060450 (+ 3548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.060454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.060456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.060556 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.063221 (+ 2665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.063224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.063226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.063422 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:29.063513 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.063833 (+ 320us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.063960 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1284}
I20260620 14:53:29.064699 14180 ts_manager.cc:194] Registered new tserver with Master: 0827ede35a8e4df79127e7cebe970b5e (127.7.24.195:41681)
I20260620 14:53:29.064800 13393 ts_manager.cc:194] Re-registered known tserver with Master: 0827ede35a8e4df79127e7cebe970b5e (127.7.24.195:41681)
I20260620 14:53:29.065317 13393 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:60867
I20260620 14:53:29.065569 14633 heartbeater.cc:347] Connected to a master server at 127.7.24.253:35457
I20260620 14:53:29.065652 14633 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:29.065831 14633 heartbeater.cc:511] Master 127.7.24.253:35457 requested a full tablet report, sending...
I20260620 14:53:29.066497 13994 ts_manager.cc:194] Registered new tserver with Master: 0827ede35a8e4df79127e7cebe970b5e (127.7.24.195:41681)
I20260620 14:53:29.065052 13390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:29.068748 13390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:29.071393 14650 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: NON_VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: true } } }
I20260620 14:53:29.071529 14650 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:29.074414 14650 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786: attempting to promote NON_VOTER 66840f8c385e4468a18e96be8573302c to VOTER
I20260620 14:53:29.074857 14650 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } }
I20260620 14:53:29.076027 14004 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 FOLLOWER]: Refusing update from remote peer d1f6ef82883544e98ed11e2e3dd8f786: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260620 14:53:29.076426 14649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260620 14:53:29.077819 14190 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 1 LEARNER]: Refusing update from remote peer d1f6ef82883544e98ed11e2e3dd8f786: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260620 14:53:29.078289 14520 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260620 14:53:29.078424 14649 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 66840f8c385e4468a18e96be8573302c (127.7.24.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:29.079653 14520 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:29.079751 14520 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:29.079687 14649 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:29.079452 14190 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 66840f8c385e4468a18e96be8573302c (127.7.24.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:29.079792 14649 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:29.080369 14004 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 66840f8c385e4468a18e96be8573302c (127.7.24.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:29.081056 14224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:29.081182 14224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:53:29.081578 14650 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:29.081696 14650 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:29.082387 14659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:29.082540 14659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:53:29.089458 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.083554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38787 (local address 127.0.0.1:44994)
0620 14:53:29.083712 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:29.083728 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:29.083806 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:29.084996 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:29.084999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:29.085016 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:29.085124 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.085130 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.086783 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.086786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.088023 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.088034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.089015 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.089019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.089104 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.089130 (+ 26us) client_negotiation.cc:769] Sending connection context
0620 14:53:29.089222 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 14:53:29.089303 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":23}
I20260620 14:53:29.089641 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.083837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:41681 (local address 127.0.0.1:35390)
0620 14:53:29.084756 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:29.084769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:29.085013 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:29.085447 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:29.085449 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:29.085464 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:29.085562 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.085567 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.086845 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.086853 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.087926 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.087939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.089258 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.089262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.089349 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.089371 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 14:53:29.089440 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 14:53:29.089500 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"thread_start_us":95,"threads_started":1}
I20260620 14:53:29.089777 14660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.083722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44994 (local address 127.7.24.194:38787)
0620 14:53:29.084463 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.084475 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.084500 (+ 25us) server_negotiation.cc:407] Connection header received
0620 14:53:29.084579 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.084586 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.084824 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.085313 (+ 489us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:29.085458 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.086676 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.088125 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.088901 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.089017 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.089195 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.089255 (+ 60us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.089349 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":108,"threads_started":1}
I20260620 14:53:29.090142 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.084097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:39179 (local address 127.0.0.1:53718)
0620 14:53:29.085225 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:29.085247 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:29.085328 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:29.085890 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:29.085893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:29.085910 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:29.086011 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.086017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.087211 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.087214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.088351 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.088359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.089468 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.089474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.089883 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.089911 (+ 28us) client_negotiation.cc:769] Sending connection context
0620 14:53:29.089984 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 14:53:29.090051 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"mutex_wait_us":24,"thread_start_us":76,"threads_started":1}
I20260620 14:53:29.090587 14664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.084571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53718 (local address 127.7.24.193:39179)
0620 14:53:29.085478 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.085493 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.085513 (+ 20us) server_negotiation.cc:407] Connection header received
0620 14:53:29.085581 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.085588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.085764 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.085901 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:29.086100 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.087078 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.088470 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.089293 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.089385 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.089783 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.090323 (+ 540us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.090413 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":62,"threads_started":1}
I20260620 14:53:29.090876 14663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.084236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.7.24.195:41681)
0620 14:53:29.084548 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.084558 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.084943 (+ 385us) server_negotiation.cc:407] Connection header received
0620 14:53:29.085147 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.085161 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.085336 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.085473 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:29.085652 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.086725 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.088041 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.088901 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.089086 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.089197 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.089889 (+ 692us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.090729 (+ 840us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":109,"threads_started":1}
I20260620 14:53:29.092981 14564 tablet_service.cc:1467] Processing CreateTablet for tablet 4b25ed4d299d4d0a89a492cc9f190bcf (DEFAULT_TABLE table=test-table [id=e9173f72b0a54b1da684914dfc5a0bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:29.092999 14427 tablet_service.cc:1467] Processing CreateTablet for tablet 4b25ed4d299d4d0a89a492cc9f190bcf (DEFAULT_TABLE table=test-table [id=e9173f72b0a54b1da684914dfc5a0bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:29.092999 14290 tablet_service.cc:1467] Processing CreateTablet for tablet 4b25ed4d299d4d0a89a492cc9f190bcf (DEFAULT_TABLE table=test-table [id=e9173f72b0a54b1da684914dfc5a0bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:29.093945 14564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b25ed4d299d4d0a89a492cc9f190bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:29.093962 14427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b25ed4d299d4d0a89a492cc9f190bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:29.093962 14290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b25ed4d299d4d0a89a492cc9f190bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:29.100332 14665 tablet_bootstrap.cc:492] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e: Bootstrap starting.
I20260620 14:53:29.101601 14666 tablet_bootstrap.cc:492] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: Bootstrap starting.
I20260620 14:53:29.102560 14667 tablet_bootstrap.cc:492] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71: Bootstrap starting.
I20260620 14:53:29.102813 14665 tablet_bootstrap.cc:654] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:29.103521 14666 tablet_bootstrap.cc:654] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:29.103765 14665 log.cc:824] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:29.104281 14666 log.cc:824] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:29.104453 14667 tablet_bootstrap.cc:654] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:29.105195 14667 log.cc:824] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:29.105806 14665 tablet_bootstrap.cc:492] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e: No bootstrap required, opened a new log
I20260620 14:53:29.105995 14665 ts_tablet_manager.cc:1397] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260620 14:53:29.106127 14666 tablet_bootstrap.cc:492] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: No bootstrap required, opened a new log
I20260620 14:53:29.106305 14666 ts_tablet_manager.cc:1397] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 14:53:29.106838 14667 tablet_bootstrap.cc:492] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71: No bootstrap required, opened a new log
I20260620 14:53:29.107013 14667 ts_tablet_manager.cc:1397] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 14:53:29.110436 14667 raft_consensus.cc:348] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:29.110728 14667 raft_consensus.cc:374] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:29.110780 14667 raft_consensus.cc:729] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0bb85f6fa2348cda24dda06e035bf71, State: Initialized, Role: FOLLOWER
I20260620 14:53:29.110670 14665 raft_consensus.cc:348] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:29.110670 14666 raft_consensus.cc:348] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:29.110910 14666 raft_consensus.cc:374] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:29.110910 14665 raft_consensus.cc:374] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:29.110970 14666 raft_consensus.cc:729] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d134ac5d57ad4d7c82e1d5d47f188909, State: Initialized, Role: FOLLOWER
I20260620 14:53:29.110970 14665 raft_consensus.cc:729] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0827ede35a8e4df79127e7cebe970b5e, State: Initialized, Role: FOLLOWER
I20260620 14:53:29.111249 14667 consensus_queue.cc:260] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:29.111373 14666 consensus_queue.cc:260] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:29.111373 14665 consensus_queue.cc:260] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:29.112030 14356 heartbeater.cc:503] Master 127.7.24.254:40517 was elected leader, sending a full tablet report...
I20260620 14:53:29.112282 14667 ts_tablet_manager.cc:1428] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260620 14:53:29.112288 14666 ts_tablet_manager.cc:1428] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260620 14:53:29.112283 14665 ts_tablet_manager.cc:1428] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260620 14:53:29.112581 14493 heartbeater.cc:503] Master 127.7.24.254:40517 was elected leader, sending a full tablet report...
I20260620 14:53:29.112579 14631 heartbeater.cc:503] Master 127.7.24.254:40517 was elected leader, sending a full tablet report...
W20260620 14:53:29.126981 14359 tablet.cc:2369] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:53:29.206949 14673 raft_consensus.cc:482] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:29.207237 14673 raft_consensus.cc:504] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:29.208532 14673 leader_election.cc:288] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0bb85f6fa2348cda24dda06e035bf71 (127.7.24.194:38787), 0827ede35a8e4df79127e7cebe970b5e (127.7.24.195:41681)
I20260620 14:53:29.214098 14660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.208965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:38243 (local address 127.7.24.194:38787)
0620 14:53:29.209092 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.209098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.209166 (+ 68us) server_negotiation.cc:407] Connection header received
0620 14:53:29.209313 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.209317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.209370 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.209452 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:29.209906 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.210733 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.212581 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.213209 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.213357 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.213414 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.213868 (+ 454us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.213947 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 14:53:29.214098 14663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.208884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:44803 (local address 127.7.24.195:41681)
0620 14:53:29.209070 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.209075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.209155 (+ 80us) server_negotiation.cc:407] Connection header received
0620 14:53:29.209313 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.209317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.209370 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.209452 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:29.209906 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.210734 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.212633 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.213248 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.213375 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.213416 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.213868 (+ 452us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.213947 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260620 14:53:29.214146 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.208790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:41681 (local address 127.7.24.193:44803)
0620 14:53:29.209064 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:29.209080 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:29.209200 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:29.209477 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:29.209480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:29.209504 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:29.209734 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.209740 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.210865 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.210868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.212429 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.212441 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.213427 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.213431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.213691 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.213747 (+ 56us) client_negotiation.cc:769] Sending connection context
0620 14:53:29.213830 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 14:53:29.213956 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":239,"spinlock_wait_cycles":1792,"thread_start_us":77,"threads_started":1}
I20260620 14:53:29.214146 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.208803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38787 (local address 127.7.24.193:38243)
0620 14:53:29.209064 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:29.209080 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:29.209200 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:29.209477 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:29.209480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:29.209506 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:29.209733 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.209740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.210865 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.210868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.212429 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:29.212441 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.213427 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.213431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:29.213686 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.213743 (+ 57us) client_negotiation.cc:769] Sending connection context
0620 14:53:29.213830 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 14:53:29.213956 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":418,"spinlock_wait_cycles":27904,"thread_start_us":63,"threads_started":1}
I20260620 14:53:29.214851 14585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b25ed4d299d4d0a89a492cc9f190bcf" candidate_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0827ede35a8e4df79127e7cebe970b5e" is_pre_election: true
I20260620 14:53:29.214985 14447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b25ed4d299d4d0a89a492cc9f190bcf" candidate_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bb85f6fa2348cda24dda06e035bf71" is_pre_election: true
I20260620 14:53:29.215212 14585 raft_consensus.cc:2454] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d134ac5d57ad4d7c82e1d5d47f188909 in term 0.
I20260620 14:53:29.215211 14447 raft_consensus.cc:2454] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d134ac5d57ad4d7c82e1d5d47f188909 in term 0.
I20260620 14:53:29.215698 14244 leader_election.cc:302] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0827ede35a8e4df79127e7cebe970b5e, d134ac5d57ad4d7c82e1d5d47f188909; no voters:
I20260620 14:53:29.215971 14673 raft_consensus.cc:2781] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:29.216055 14673 raft_consensus.cc:482] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:29.216105 14673 raft_consensus.cc:3037] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:29.217114 14673 raft_consensus.cc:504] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:29.217445 14673 leader_election.cc:288] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [CANDIDATE]: Term 1 election: Requested vote from peers a0bb85f6fa2348cda24dda06e035bf71 (127.7.24.194:38787), 0827ede35a8e4df79127e7cebe970b5e (127.7.24.195:41681)
I20260620 14:53:29.217779 14585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b25ed4d299d4d0a89a492cc9f190bcf" candidate_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0827ede35a8e4df79127e7cebe970b5e"
I20260620 14:53:29.217818 14447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b25ed4d299d4d0a89a492cc9f190bcf" candidate_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bb85f6fa2348cda24dda06e035bf71"
I20260620 14:53:29.217914 14585 raft_consensus.cc:3037] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:29.217924 14447 raft_consensus.cc:3037] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:29.219223 14447 raft_consensus.cc:2454] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d134ac5d57ad4d7c82e1d5d47f188909 in term 1.
I20260620 14:53:29.219223 14585 raft_consensus.cc:2454] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d134ac5d57ad4d7c82e1d5d47f188909 in term 1.
I20260620 14:53:29.219530 14244 leader_election.cc:302] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0827ede35a8e4df79127e7cebe970b5e, d134ac5d57ad4d7c82e1d5d47f188909; no voters:
I20260620 14:53:29.219720 14673 raft_consensus.cc:2781] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:29.219926 14673 raft_consensus.cc:686] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 1 LEADER]: Becoming Leader. State: Replica: d134ac5d57ad4d7c82e1d5d47f188909, State: Running, Role: LEADER
I20260620 14:53:29.220299 14673 consensus_queue.cc:237] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:29.223690 13392 catalog_manager.cc:5495] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 reported cstate change: term changed from 0 to 1, leader changed from <none> to d134ac5d57ad4d7c82e1d5d47f188909 (127.7.24.193). New cstate: current_term: 1 leader_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } health_report { overall_health: HEALTHY } } }
I20260620 14:53:29.245904 14664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.242047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53734 (local address 127.7.24.193:39179)
0620 14:53:29.242203 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.242207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.242222 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:53:29.242382 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.242384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.242528 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.242640 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:29.242958 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.243505 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.244377 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.244638 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.244718 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.245028 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:29.245471 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:29.245619 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.245658 (+ 39us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.245732 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 14:53:29.252292 14447 raft_consensus.cc:1261] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71 [term 1 FOLLOWER]: Refusing update from remote peer d134ac5d57ad4d7c82e1d5d47f188909: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:29.253007 14673 consensus_queue.cc:1035] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:29.253089 14585 raft_consensus.cc:1261] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e [term 1 FOLLOWER]: Refusing update from remote peer d134ac5d57ad4d7c82e1d5d47f188909: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:29.253682 14673 consensus_queue.cc:1035] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:29.269196 14680 mvcc.cc:205] Tried to move back new op lower bound from 7298937689090060288 to 7298937688968798208. Current Snapshot: MvccSnapshot[applied={T|T < 7298937689090060288}]
I20260620 14:53:29.271490 13392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:29.271658 13392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:29.276010 13392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1f6ef82883544e98ed11e2e3dd8f786: Sending DeleteTablet for 3 replicas of tablet 4b25ed4d299d4d0a89a492cc9f190bcf
I20260620 14:53:29.276847 14427 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b25ed4d299d4d0a89a492cc9f190bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:29 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:44994
I20260620 14:53:29.276854 14564 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b25ed4d299d4d0a89a492cc9f190bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:29 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:35390
I20260620 14:53:29.277449 14689 tablet_replica.cc:330] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e: stopping tablet replica
I20260620 14:53:29.277479 14290 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b25ed4d299d4d0a89a492cc9f190bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:29 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:53718
I20260620 14:53:29.277925 14689 raft_consensus.cc:2229] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:29.277949 14691 tablet_replica.cc:330] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: stopping tablet replica
I20260620 14:53:29.278204 14691 raft_consensus.cc:2229] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:29.278396 14689 raft_consensus.cc:2258] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:29.278597 14691 raft_consensus.cc:2258] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:29.279754 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 13361
I20260620 14:53:29.281859 14691 ts_tablet_manager.cc:1905] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:29.281956 14689 ts_tablet_manager.cc:1905] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:29.285148 14691 ts_tablet_manager.cc:1918] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:29.285249 14691 log.cc:1196] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-0/wal/wals/4b25ed4d299d4d0a89a492cc9f190bcf
I20260620 14:53:29.285512 14689 ts_tablet_manager.cc:1918] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:29.285611 14689 log.cc:1196] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/4b25ed4d299d4d0a89a492cc9f190bcf
I20260620 14:53:29.285705 14691 ts_tablet_manager.cc:1939] T 4b25ed4d299d4d0a89a492cc9f190bcf P d134ac5d57ad4d7c82e1d5d47f188909: Deleting consensus metadata
I20260620 14:53:29.285941 14689 ts_tablet_manager.cc:1939] T 4b25ed4d299d4d0a89a492cc9f190bcf P 0827ede35a8e4df79127e7cebe970b5e: Deleting consensus metadata
I20260620 14:53:29.287276 14690 tablet_replica.cc:330] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71: stopping tablet replica
W20260620 14:53:29.287655 14517 connection.cc:535] server connection from 127.0.0.1:35390 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 14:53:29.287878 14690 raft_consensus.cc:2229] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:29.288297 14690 raft_consensus.cc:2258] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:29.290005 14690 ts_tablet_manager.cc:1905] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 14:53:29.291484 14244 connection.cc:441] server connection from 127.0.0.1:53718 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53718 (request call id 1) could send its response
I20260620 14:53:29.292923 14690 ts_tablet_manager.cc:1918] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:29.293032 14690 log.cc:1196] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/4b25ed4d299d4d0a89a492cc9f190bcf
I20260620 14:53:29.293401 14690 ts_tablet_manager.cc:1939] T 4b25ed4d299d4d0a89a492cc9f190bcf P a0bb85f6fa2348cda24dda06e035bf71: Deleting consensus metadata
W20260620 14:53:29.294610 14382 connection.cc:441] server connection from 127.0.0.1:44994 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44994 (request call id 1) could send its response
I20260620 14:53:29.299770 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.279034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60488 (local address 127.7.24.253:35457)
0620 14:53:29.279622 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.279627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.279641 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:29.279680 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.279682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.279728 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.279810 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:29.288674 (+ 8864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.289647 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.291787 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.292193 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.292280 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.295025 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:29.295029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:29.295032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:29.295065 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:29.297079 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.297645 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.297649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.297651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.297760 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.298080 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.298083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.298085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.298312 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:29.298437 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.298858 (+ 421us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.299317 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497}
I20260620 14:53:29.304642 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.292591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.7.24.252:36333)
0620 14:53:29.292723 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.292729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.294402 (+ 1673us) server_negotiation.cc:407] Connection header received
0620 14:53:29.294637 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.294641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.294697 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.294753 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:29.296096 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.296909 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.298118 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.298507 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.298584 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.300439 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:29.300444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:29.300449 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:29.300485 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:29.302540 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.303049 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.303053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.303055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.303162 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.303465 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.303469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.303471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.303808 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:29.303933 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.304297 (+ 364us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.304426 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 14:53:29.304881 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.293489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.7.24.253:35457)
0620 14:53:29.293631 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.293635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.295636 (+ 2001us) server_negotiation.cc:407] Connection header received
0620 14:53:29.295753 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.295756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.295828 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.295901 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:29.296701 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.297448 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.298579 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.298910 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.298997 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.301046 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:29.301057 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:29.301060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:29.301084 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:29.303103 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.303572 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.303575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.303577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.303652 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.303961 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.303967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.303969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.304185 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:29.304280 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.304609 (+ 329us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.304733 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 14:53:29.311312 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:29.279441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52552 (local address 127.7.24.252:36333)
0620 14:53:29.279541 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:29.279546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:29.301771 (+ 22225us) server_negotiation.cc:407] Connection header received
0620 14:53:29.301943 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:29.301947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:29.301995 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:29.302047 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:29.302980 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.303875 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.305135 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:29.305458 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:29.305572 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:29.307828 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:29.307831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:29.307833 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:29.307862 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:29.309410 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.309899 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.309902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.309903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.309980 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:29.310287 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:29.310290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:29.310292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:29.310561 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:29.310715 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:29.311022 (+ 307us) server_negotiation.cc:299] Negotiation successful
0620 14:53:29.311149 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 14:53:30.264736 14700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.264233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.195:36705)
0620 14:53:30.264486 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.264617 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":88,"threads_started":1}
W20260620 14:53:30.265086 14631 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:40517 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
I20260620 14:53:30.269610 14701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.269153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.193:33505)
0620 14:53:30.269404 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.269500 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":86,"threads_started":1}
I20260620 14:53:30.269811 14702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.269234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.194:34543)
0620 14:53:30.269616 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.269709 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":73,"threads_started":1}
W20260620 14:53:30.270113 14493 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:40517 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
W20260620 14:53:30.270582 14356 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:40517 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
I20260620 14:53:30.776899 14703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1f6ef82883544e98ed11e2e3dd8f786)
I20260620 14:53:30.777181 14703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } }
I20260620 14:53:30.777202 14704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1f6ef82883544e98ed11e2e3dd8f786)
I20260620 14:53:30.777473 14704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } }
I20260620 14:53:30.778266 14703 leader_election.cc:288] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1f6ef82883544e98ed11e2e3dd8f786 (127.7.24.254:40517), 66840f8c385e4468a18e96be8573302c (127.7.24.252:36333)
I20260620 14:53:30.779162 14704 leader_election.cc:288] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1f6ef82883544e98ed11e2e3dd8f786 (127.7.24.254:40517), 8838366add4347deb4f7275bd4f67ce1 (127.7.24.253:35457)
I20260620 14:53:30.779206 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.778568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55640)
0620 14:53:30.778977 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.779069 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":84,"threads_started":1}
W20260620 14:53:30.779398 13981 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
I20260620 14:53:30.780098 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.778952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55652)
0620 14:53:30.779891 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.779990 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":190,"thread_start_us":70,"threads_started":1}
W20260620 14:53:30.780391 14164 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
I20260620 14:53:30.780426 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.780127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55658)
0620 14:53:30.780274 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.780328 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260620 14:53:30.780540 13981 leader_election.cc:334] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1f6ef82883544e98ed11e2e3dd8f786 (127.7.24.254:40517): Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
I20260620 14:53:30.781260 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.781016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55660)
0620 14:53:30.781126 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.781182 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 14:53:30.781379 14164 leader_election.cc:334] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1f6ef82883544e98ed11e2e3dd8f786 (127.7.24.254:40517): Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
I20260620 14:53:30.783756 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.778821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.7.24.252:36333)
0620 14:53:30.779092 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:30.779097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:30.779409 (+ 312us) server_negotiation.cc:407] Connection header received
0620 14:53:30.779526 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:30.779530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:30.779571 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:30.779654 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:30.780112 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.781174 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.782439 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.783155 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.783277 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.783389 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:30.783572 (+ 183us) server_negotiation.cc:299] Negotiation successful
0620 14:53:30.783639 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":199,"thread_start_us":82,"threads_started":1}
I20260620 14:53:30.783782 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.778591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:36333 (local address 127.0.0.1:52564)
0620 14:53:30.779113 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.779133 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:30.779283 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:30.779814 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:30.779817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:30.779840 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:30.779968 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.779979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.781317 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.781321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.782277 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.782288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.783297 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.783300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.783418 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.783449 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 14:53:30.783549 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 14:53:30.783628 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":89,"threads_started":1}
I20260620 14:53:30.784353 14190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8838366add4347deb4f7275bd4f67ce1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "66840f8c385e4468a18e96be8573302c" is_pre_election: true
I20260620 14:53:30.784650 14190 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8838366add4347deb4f7275bd4f67ce1 in term 1.
I20260620 14:53:30.785039 14709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.778952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35457 (local address 127.0.0.1:60498)
0620 14:53:30.779407 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.779421 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:30.779551 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:30.780267 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:30.780271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:30.780299 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:30.780488 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.780501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.781716 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.781721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.783291 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.783299 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.784260 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.784265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.784542 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.784610 (+ 68us) client_negotiation.cc:769] Sending connection context
0620 14:53:30.784722 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 14:53:30.784861 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":27,"thread_start_us":60,"threads_started":1}
I20260620 14:53:30.785055 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.779492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60498 (local address 127.7.24.253:35457)
0620 14:53:30.779925 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:30.779930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:30.779948 (+ 18us) server_negotiation.cc:407] Connection header received
0620 14:53:30.780014 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:30.780018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:30.780079 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:30.780457 (+ 378us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:30.780807 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.781602 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.783412 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.784082 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.784246 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.784303 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:30.784776 (+ 473us) server_negotiation.cc:299] Negotiation successful
0620 14:53:30.784907 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":66,"threads_started":1}
I20260620 14:53:30.785107 13980 leader_election.cc:302] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66840f8c385e4468a18e96be8573302c, 8838366add4347deb4f7275bd4f67ce1; no voters: d1f6ef82883544e98ed11e2e3dd8f786
I20260620 14:53:30.785358 14703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 14:53:30.785426 14703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1f6ef82883544e98ed11e2e3dd8f786)
I20260620 14:53:30.785465 14703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 1 FOLLOWER]: Advancing to term 2
I20260620 14:53:30.785876 14004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66840f8c385e4468a18e96be8573302c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8838366add4347deb4f7275bd4f67ce1" is_pre_election: true
I20260620 14:53:30.786907 14703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } }
I20260620 14:53:30.787266 14004 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 66840f8c385e4468a18e96be8573302c in current term 2: Already voted for candidate 8838366add4347deb4f7275bd4f67ce1 in this term.
I20260620 14:53:30.787372 14703 leader_election.cc:288] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [CANDIDATE]: Term 2 election: Requested vote from peers d1f6ef82883544e98ed11e2e3dd8f786 (127.7.24.254:40517), 66840f8c385e4468a18e96be8573302c (127.7.24.252:36333)
I20260620 14:53:30.787698 14190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8838366add4347deb4f7275bd4f67ce1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "66840f8c385e4468a18e96be8573302c"
I20260620 14:53:30.787854 14190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 1 FOLLOWER]: Advancing to term 2
I20260620 14:53:30.788314 14162 leader_election.cc:302] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66840f8c385e4468a18e96be8573302c; no voters: 8838366add4347deb4f7275bd4f67ce1, d1f6ef82883544e98ed11e2e3dd8f786
I20260620 14:53:30.789373 14190 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8838366add4347deb4f7275bd4f67ce1 in term 2.
I20260620 14:53:30.789526 14704 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 14:53:30.789710 13980 leader_election.cc:302] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66840f8c385e4468a18e96be8573302c, 8838366add4347deb4f7275bd4f67ce1; no voters:
I20260620 14:53:30.789839 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.787538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55664)
0620 14:53:30.789633 (+ 2095us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.789703 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2012}
I20260620 14:53:30.789955 14703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 14:53:30.790246 14703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [term 2 LEADER]: Becoming Leader. State: Replica: 8838366add4347deb4f7275bd4f67ce1, State: Running, Role: LEADER
I20260620 14:53:30.790684 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.790448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55680)
0620 14:53:30.790587 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.790640 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260620 14:53:30.790603 14703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } }
W20260620 14:53:30.790818 13981 leader_election.cc:334] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d1f6ef82883544e98ed11e2e3dd8f786 (127.7.24.254:40517): Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
I20260620 14:53:30.791878 14713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8838366add4347deb4f7275bd4f67ce1. Latest consensus state: current_term: 2 leader_uuid: "8838366add4347deb4f7275bd4f67ce1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:30.791988 14713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:30.792282 14715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:30.794389 14715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:30.794870 14715 catalog_manager.cc:1259] Loaded cluster ID: eb31eea6bdc4446dbf29f409f6d52a5d
I20260620 14:53:30.794921 14715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:30.795507 14715 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:30.795928 14715 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: Loaded TSK: 0
I20260620 14:53:30.796633 14715 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:53:30.826828 13993 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:30.830022 13993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:30.834079 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.833722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55692)
0620 14:53:30.833882 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.833969 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260620 14:53:30.834221 14190 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [term 2 FOLLOWER]: Refusing update from remote peer 8838366add4347deb4f7275bd4f67ce1: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260620 14:53:30.834947 14713 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260620 14:53:30.835179 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.834848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.0.0.1:55698)
0620 14:53:30.835014 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.835070 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260620 14:53:30.835459 13981 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 -> Peer d1f6ef82883544e98ed11e2e3dd8f786 (127.7.24.254:40517): Couldn't send request to peer d1f6ef82883544e98ed11e2e3dd8f786. Status: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 14:53:30.836766 14704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8838366add4347deb4f7275bd4f67ce1. Latest consensus state: current_term: 2 leader_uuid: "8838366add4347deb4f7275bd4f67ce1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:30.837013 14704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:53:30.838238 14703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "8838366add4347deb4f7275bd4f67ce1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:30.838407 14703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:30.838819 14704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "8838366add4347deb4f7275bd4f67ce1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1f6ef82883544e98ed11e2e3dd8f786" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40517 } } peers { permanent_uuid: "8838366add4347deb4f7275bd4f67ce1" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35457 } attrs { promote: false } } peers { permanent_uuid: "66840f8c385e4468a18e96be8573302c" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 36333 } attrs { promote: false } } }
I20260620 14:53:30.838989 14704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66840f8c385e4468a18e96be8573302c [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:53:30.850904 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.845804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:41681 (local address 127.0.0.1:35394)
0620 14:53:30.845970 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.845986 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:30.846073 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:30.846822 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:30.846824 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:30.846838 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:30.847002 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.847013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.848273 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.848276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.849583 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.849594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.850409 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.850413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.850537 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.850560 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 14:53:30.850661 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 14:53:30.850753 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":53}
I20260620 14:53:30.850905 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.845974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35394 (local address 127.7.24.195:41681)
0620 14:53:30.846524 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:30.846532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:30.846551 (+ 19us) server_negotiation.cc:407] Connection header received
0620 14:53:30.846613 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:30.846618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:30.846682 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:30.846819 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:30.847175 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.847802 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.849739 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.850280 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.850395 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.850468 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:30.850678 (+ 210us) server_negotiation.cc:299] Negotiation successful
0620 14:53:30.850756 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":124,"threads_started":1}
I20260620 14:53:30.851333 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.846305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:39179 (local address 127.0.0.1:53748)
0620 14:53:30.846980 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.846997 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:30.847131 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:30.847455 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:30.847458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:30.847476 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:30.847612 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.847618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.848829 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.848833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.849771 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.849780 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.850739 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.850743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.850906 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.850972 (+ 66us) client_negotiation.cc:769] Sending connection context
0620 14:53:30.851034 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 14:53:30.851157 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":5,"thread_start_us":161,"threads_started":1}
I20260620 14:53:30.851617 14564 tablet_service.cc:1467] Processing CreateTablet for tablet 8099ca88268c4b08ac6732d9031a0150 (DEFAULT_TABLE table=test-table [id=a5da3db752b5423ab907cf71da37a4af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:30.851918 14564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8099ca88268c4b08ac6732d9031a0150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:30.852626 14723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.846608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53748 (local address 127.7.24.193:39179)
0620 14:53:30.846958 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:30.846963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:30.847086 (+ 123us) server_negotiation.cc:407] Connection header received
0620 14:53:30.847250 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:30.847254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:30.847314 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:30.847716 (+ 402us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:30.847835 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.848695 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.849910 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.850593 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.852221 (+ 1628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.852319 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:30.852359 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 14:53:30.852459 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":114,"threads_started":1}
I20260620 14:53:30.853050 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.846750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.7.24.194:38787)
0620 14:53:30.847387 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:30.847393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:30.847410 (+ 17us) server_negotiation.cc:407] Connection header received
0620 14:53:30.847471 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:30.847474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:30.847523 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:30.847618 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:30.847999 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.848506 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.850086 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.850798 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.851131 (+ 333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.851207 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:30.852761 (+ 1554us) server_negotiation.cc:299] Negotiation successful
0620 14:53:30.852900 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":88,"threads_started":1}
I20260620 14:53:30.853363 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.846058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38787 (local address 127.0.0.1:45006)
0620 14:53:30.846605 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.846620 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:30.846713 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:30.847700 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:30.847704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:30.847725 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:30.847815 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.847821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.848624 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.848628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.849939 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.849948 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.851271 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.851275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.851473 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.851506 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 14:53:30.851601 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 14:53:30.853244 (+ 1643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":52}
I20260620 14:53:30.854003 14427 tablet_service.cc:1467] Processing CreateTablet for tablet 8099ca88268c4b08ac6732d9031a0150 (DEFAULT_TABLE table=test-table [id=a5da3db752b5423ab907cf71da37a4af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:30.854372 14427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8099ca88268c4b08ac6732d9031a0150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:30.854444 14725 tablet_bootstrap.cc:492] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e: Bootstrap starting.
I20260620 14:53:30.854837 14290 tablet_service.cc:1467] Processing CreateTablet for tablet 8099ca88268c4b08ac6732d9031a0150 (DEFAULT_TABLE table=test-table [id=a5da3db752b5423ab907cf71da37a4af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:30.855177 14290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8099ca88268c4b08ac6732d9031a0150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:30.855549 14725 tablet_bootstrap.cc:654] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:30.857554 14728 tablet_bootstrap.cc:492] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909: Bootstrap starting.
I20260620 14:53:30.857599 14727 tablet_bootstrap.cc:492] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71: Bootstrap starting.
I20260620 14:53:30.858760 14725 tablet_bootstrap.cc:492] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e: No bootstrap required, opened a new log
I20260620 14:53:30.858783 14727 tablet_bootstrap.cc:654] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:30.858889 14725 ts_tablet_manager.cc:1397] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260620 14:53:30.859359 14725 raft_consensus.cc:348] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:30.859500 14725 raft_consensus.cc:374] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:30.859545 14725 raft_consensus.cc:729] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0827ede35a8e4df79127e7cebe970b5e, State: Initialized, Role: FOLLOWER
I20260620 14:53:30.859802 14725 consensus_queue.cc:260] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:30.860277 14725 ts_tablet_manager.cc:1428] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:30.860865 14728 tablet_bootstrap.cc:654] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:30.862118 14728 tablet_bootstrap.cc:492] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909: No bootstrap required, opened a new log
I20260620 14:53:30.862217 14728 ts_tablet_manager.cc:1397] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260620 14:53:30.862319 14727 tablet_bootstrap.cc:492] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71: No bootstrap required, opened a new log
I20260620 14:53:30.862432 14727 ts_tablet_manager.cc:1397] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260620 14:53:30.862675 14728 raft_consensus.cc:348] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:30.862795 14728 raft_consensus.cc:374] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:30.862826 14728 raft_consensus.cc:729] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d134ac5d57ad4d7c82e1d5d47f188909, State: Initialized, Role: FOLLOWER
I20260620 14:53:30.862951 14728 consensus_queue.cc:260] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:30.863318 14728 ts_tablet_manager.cc:1428] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:30.863341 14633 heartbeater.cc:503] Master 127.7.24.253:35457 was elected leader, sending a full tablet report...
I20260620 14:53:30.863423 14727 raft_consensus.cc:348] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:30.863576 14727 raft_consensus.cc:374] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:30.863616 14727 raft_consensus.cc:729] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0bb85f6fa2348cda24dda06e035bf71, State: Initialized, Role: FOLLOWER
I20260620 14:53:30.863778 14727 consensus_queue.cc:260] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:30.864161 14727 ts_tablet_manager.cc:1428] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:30.864310 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.863694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.193:54029)
0620 14:53:30.864047 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.864128 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":111,"threads_started":1}
I20260620 14:53:30.865260 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.864848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.193:46801)
0620 14:53:30.865133 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.865180 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":98,"threads_started":1}
I20260620 14:53:30.866534 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.864600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.194:36307)
0620 14:53:30.866261 (+ 1661us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.866344 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1545,"thread_start_us":106,"threads_started":1}
I20260620 14:53:30.866999 14731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.860668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.195:36803)
0620 14:53:30.866825 (+ 6157us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.866893 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6080,"thread_start_us":126,"threads_started":1}
I20260620 14:53:30.867654 14495 heartbeater.cc:503] Master 127.7.24.253:35457 was elected leader, sending a full tablet report...
I20260620 14:53:30.868197 14731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.867525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.195:47091)
0620 14:53:30.868053 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.868111 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437}
I20260620 14:53:30.868173 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.867779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.194:55863)
0620 14:53:30.868048 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.868107 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260620 14:53:30.900112 14735 raft_consensus.cc:482] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:30.900280 14735 raft_consensus.cc:504] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:30.901364 14735 leader_election.cc:288] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0827ede35a8e4df79127e7cebe970b5e (127.7.24.195:41681), d134ac5d57ad4d7c82e1d5d47f188909 (127.7.24.193:39179)
I20260620 14:53:30.905969 14721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.901916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:37951 (local address 127.7.24.195:41681)
0620 14:53:30.902049 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:30.902054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:30.902376 (+ 322us) server_negotiation.cc:407] Connection header received
0620 14:53:30.902585 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:30.902588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:30.902639 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:30.902729 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:30.903055 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.903598 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.904960 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.905361 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.905514 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.905575 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:30.905763 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 14:53:30.905837 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 14:53:30.906168 14723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.901904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:50297 (local address 127.7.24.193:39179)
0620 14:53:30.902049 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:30.902054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:30.902239 (+ 185us) server_negotiation.cc:407] Connection header received
0620 14:53:30.902502 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:30.902505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:30.902568 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:30.902678 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:30.903067 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.903649 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.904959 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.905419 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.905574 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.905653 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:30.905857 (+ 204us) server_negotiation.cc:299] Negotiation successful
0620 14:53:30.905968 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 14:53:30.905990 14738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.901835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:41681 (local address 127.7.24.194:37951)
0620 14:53:30.902248 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.902263 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:30.902437 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:30.902751 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:30.902754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:30.902771 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:30.902949 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.902955 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.903718 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.903725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.904833 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.904842 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.905514 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.905518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.905620 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.905654 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 14:53:30.905741 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 14:53:30.905827 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":57,"thread_start_us":174,"threads_started":1}
I20260620 14:53:30.906875 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.901797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:39179 (local address 127.7.24.194:50297)
0620 14:53:30.902086 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.902107 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:30.902317 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:30.902692 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:30.902696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:30.902717 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:30.902939 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.902946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.903776 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.903780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.904816 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:30.904826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.905569 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.905574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:30.905700 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.905725 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 14:53:30.905836 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 14:53:30.905940 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":91}
I20260620 14:53:30.907433 14585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8099ca88268c4b08ac6732d9031a0150" candidate_uuid: "a0bb85f6fa2348cda24dda06e035bf71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0827ede35a8e4df79127e7cebe970b5e" is_pre_election: true
I20260620 14:53:30.907501 14310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8099ca88268c4b08ac6732d9031a0150" candidate_uuid: "a0bb85f6fa2348cda24dda06e035bf71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" is_pre_election: true
I20260620 14:53:30.907634 14585 raft_consensus.cc:2454] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0bb85f6fa2348cda24dda06e035bf71 in term 0.
I20260620 14:53:30.907805 14310 raft_consensus.cc:2454] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0bb85f6fa2348cda24dda06e035bf71 in term 0.
I20260620 14:53:30.908156 14381 leader_election.cc:302] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0827ede35a8e4df79127e7cebe970b5e, a0bb85f6fa2348cda24dda06e035bf71; no voters:
I20260620 14:53:30.908668 14735 raft_consensus.cc:2781] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:30.908753 14735 raft_consensus.cc:482] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:30.908808 14735 raft_consensus.cc:3037] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:30.909955 14735 raft_consensus.cc:504] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:30.910385 14735 leader_election.cc:288] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [CANDIDATE]: Term 1 election: Requested vote from peers 0827ede35a8e4df79127e7cebe970b5e (127.7.24.195:41681), d134ac5d57ad4d7c82e1d5d47f188909 (127.7.24.193:39179)
I20260620 14:53:30.910884 14310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8099ca88268c4b08ac6732d9031a0150" candidate_uuid: "a0bb85f6fa2348cda24dda06e035bf71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d134ac5d57ad4d7c82e1d5d47f188909"
I20260620 14:53:30.910874 14585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8099ca88268c4b08ac6732d9031a0150" candidate_uuid: "a0bb85f6fa2348cda24dda06e035bf71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0827ede35a8e4df79127e7cebe970b5e"
I20260620 14:53:30.910998 14310 raft_consensus.cc:3037] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:30.911039 14585 raft_consensus.cc:3037] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:30.912014 14310 raft_consensus.cc:2454] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0bb85f6fa2348cda24dda06e035bf71 in term 1.
I20260620 14:53:30.912281 14585 raft_consensus.cc:2454] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0bb85f6fa2348cda24dda06e035bf71 in term 1.
I20260620 14:53:30.912338 14382 leader_election.cc:302] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0bb85f6fa2348cda24dda06e035bf71, d134ac5d57ad4d7c82e1d5d47f188909; no voters:
I20260620 14:53:30.912510 14735 raft_consensus.cc:2781] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:30.912772 14735 raft_consensus.cc:686] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 1 LEADER]: Becoming Leader. State: Replica: a0bb85f6fa2348cda24dda06e035bf71, State: Running, Role: LEADER
I20260620 14:53:30.913295 14735 consensus_queue.cc:237] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } } peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } }
I20260620 14:53:30.913728 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.913418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.194:37279)
0620 14:53:30.913572 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.913647 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260620 14:53:30.916312 13993 catalog_manager.cc:5495] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0bb85f6fa2348cda24dda06e035bf71 (127.7.24.194). New cstate: current_term: 1 leader_uuid: "a0bb85f6fa2348cda24dda06e035bf71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0bb85f6fa2348cda24dda06e035bf71" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 } health_report { overall_health: UNKNOWN } } }
I20260620 14:53:30.933818 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.929398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.7.24.194:38787)
0620 14:53:30.929570 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:30.929574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:30.929589 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:53:30.929694 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:30.929697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:30.929764 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:30.929880 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:30.930214 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.930809 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.931764 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:30.932033 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:30.932133 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:30.932439 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:30.933235 (+ 796us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:30.933373 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:30.933492 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 14:53:30.933613 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 14:53:30.937736 14585 raft_consensus.cc:1261] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e [term 1 FOLLOWER]: Refusing update from remote peer a0bb85f6fa2348cda24dda06e035bf71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:30.937873 14310 raft_consensus.cc:1261] T 8099ca88268c4b08ac6732d9031a0150 P d134ac5d57ad4d7c82e1d5d47f188909 [term 1 FOLLOWER]: Refusing update from remote peer a0bb85f6fa2348cda24dda06e035bf71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:30.938406 14739 consensus_queue.cc:1035] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0827ede35a8e4df79127e7cebe970b5e" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:30.938725 14735 consensus_queue.cc:1035] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d134ac5d57ad4d7c82e1d5d47f188909" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 39179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:30.940327 14358 heartbeater.cc:503] Master 127.7.24.253:35457 was elected leader, sending a full tablet report...
I20260620 14:53:30.941123 14731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.940667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.195:60123)
0620 14:53:30.940839 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.940907 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260620 14:53:30.941125 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.940671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.193:49733)
0620 14:53:30.940838 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.940908 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260620 14:53:30.944098 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.943764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.194:44045)
0620 14:53:30.943939 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.944006 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260620 14:53:30.947088 14731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.946589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.195:43047)
0620 14:53:30.946946 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.947007 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288}
I20260620 14:53:30.947690 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.947417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.194:57617)
0620 14:53:30.947560 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.947618 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260620 14:53:30.948529 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:30.948233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40517 (local address 127.7.24.193:60787)
0620 14:53:30.948394 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.948445 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260620 14:53:30.949712 14744 mvcc.cc:205] Tried to move back new op lower bound from 7298937695994707968 to 7298937695903895552. Current Snapshot: MvccSnapshot[applied={T|T < 7298937695994707968}]
I20260620 14:53:30.951152 14745 mvcc.cc:205] Tried to move back new op lower bound from 7298937695994707968 to 7298937695903895552. Current Snapshot: MvccSnapshot[applied={T|T < 7298937695994707968 or (T in {7298937695994707968})}]
I20260620 14:53:30.952762 13993 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:30.952973 13993 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:30.957571 13993 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8838366add4347deb4f7275bd4f67ce1: Sending DeleteTablet for 3 replicas of tablet 8099ca88268c4b08ac6732d9031a0150
I20260620 14:53:30.958540 14564 tablet_service.cc:1514] Processing DeleteTablet for tablet 8099ca88268c4b08ac6732d9031a0150 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:30 UTC) from {username='oryx', principal='oryx/127.7.24.253@KRBTEST.COM'} at 127.0.0.1:35394
I20260620 14:53:30.958673 14427 tablet_service.cc:1514] Processing DeleteTablet for tablet 8099ca88268c4b08ac6732d9031a0150 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:30 UTC) from {username='oryx', principal='oryx/127.7.24.253@KRBTEST.COM'} at 127.0.0.1:45006
I20260620 14:53:30.958966 14290 tablet_service.cc:1514] Processing DeleteTablet for tablet 8099ca88268c4b08ac6732d9031a0150 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:30 UTC) from {username='oryx', principal='oryx/127.7.24.253@KRBTEST.COM'} at 127.0.0.1:53748
I20260620 14:53:30.959036 14752 tablet_replica.cc:330] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e: stopping tablet replica
I20260620 14:53:30.959188 14752 raft_consensus.cc:2229] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:30.959352 14752 raft_consensus.cc:2258] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:30.959609 14753 tablet_replica.cc:330] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71: stopping tablet replica
I20260620 14:53:30.959878 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 14138
I20260620 14:53:30.959888 14753 raft_consensus.cc:2229] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:30.961285 14753 raft_consensus.cc:2258] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:30.962178 14753 ts_tablet_manager.cc:1905] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:30.964910 14753 ts_tablet_manager.cc:1918] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:30.964985 14753 log.cc:1196] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/8099ca88268c4b08ac6732d9031a0150
I20260620 14:53:30.965224 14753 ts_tablet_manager.cc:1939] T 8099ca88268c4b08ac6732d9031a0150 P a0bb85f6fa2348cda24dda06e035bf71: Deleting consensus metadata
I20260620 14:53:30.965981 13980 catalog_manager.cc:4841] TS a0bb85f6fa2348cda24dda06e035bf71 (127.7.24.194:38787): tablet 8099ca88268c4b08ac6732d9031a0150 (table test-table [id=a5da3db752b5423ab907cf71da37a4af]) successfully deleted
I20260620 14:53:30.967700 14752 ts_tablet_manager.cc:1905] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:30.970225 14752 ts_tablet_manager.cc:1918] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:30.970330 14752 log.cc:1196] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/8099ca88268c4b08ac6732d9031a0150
I20260620 14:53:30.970702 14752 ts_tablet_manager.cc:1939] T 8099ca88268c4b08ac6732d9031a0150 P 0827ede35a8e4df79127e7cebe970b5e: Deleting consensus metadata
I20260620 14:53:30.971626 13979 catalog_manager.cc:4841] TS 0827ede35a8e4df79127e7cebe970b5e (127.7.24.195:41681): tablet 8099ca88268c4b08ac6732d9031a0150 (table test-table [id=a5da3db752b5423ab907cf71da37a4af]) successfully deleted
W20260620 14:53:30.972402 14722 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:30.971971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:39179 (local address 127.0.0.1:53760)
0620 14:53:30.972161 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:30.972234 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:39179: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":51}
W20260620 14:53:30.972718 13980 catalog_manager.cc:4584] TS d134ac5d57ad4d7c82e1d5d47f188909 (127.7.24.193:39179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8099ca88268c4b08ac6732d9031a0150: Network error: Client connection negotiation failed: client connection to 127.7.24.193:39179: connect: Connection reset by peer (error 104)
I20260620 14:53:30.973155 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 14364
I20260620 14:53:30.981685 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 14500
I20260620 14:53:30.988981 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 13857
I20260620 14:53:30.996320 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 13958
2026-06-20T14:53:31Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5376 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:31 dist-test-slave-sms9 krb5kdc[14757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:31 dist-test-slave-sms9 krb5kdc[14757](info): set up 2 sockets
Jun 20 14:53:31 dist-test-slave-sms9 krb5kdc[14757](info): commencing operation
krb5kdc: starting...
W20260620 14:53:32.063117 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:32 dist-test-slave-sms9 krb5kdc[14757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:32Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.254.keytab.
Entry for principal oryx/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.254.keytab.
I20260620 14:53:32.190594 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:38593
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.254.keytab
--principal=oryx/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:38593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:32.303572 14770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:32.303833 14770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:32.303874 14770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:32.307821 14770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:32.307873 14770 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 14:53:32.307889 14770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:32.307909 14770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:32.307924 14770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:32.307940 14770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:32.313079 14770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:38593
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.254.keytab
--principal=oryx/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:38593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:32.314042 14770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:32.315061 14770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:32.323088 14778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:32.323141 14776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:32.323401 14770 server_base.cc:1034] running on GCE node
W20260620 14:53:32.323083 14775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:32.324050 14770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:32.325099 14770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:32.326294 14770 hybrid_clock.cc:648] HybridClock initialized: now 1781967212326284 us; error 30 us; skew 500 ppm
Jun 20 14:53:32 dist-test-slave-sms9 krb5kdc[14757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967212, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:32.329022 14770 init.cc:377] Logged in from keytab as oryx/127.7.24.254@KRBTEST.COM (short username oryx)
I20260620 14:53:32.329880 14770 webserver.cc:460] Webserver started at http://127.7.24.254:32985/ using document root <none> and password file <none>
I20260620 14:53:32.330437 14770 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:32.330581 14770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:32.330811 14770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:32.332809 14770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "0bfcdf28c056408a9ff6c748d7d7f3b4"
format_stamp: "Formatted at 2026-06-20 14:53:32 on dist-test-slave-sms9"
server_key: "5d310201ea54792618c941673a375f43"
server_key_iv: "1a49d79106be694e2dde1127cf0b7193"
server_key_version: "encryptionkey@0"
I20260620 14:53:32.333396 14770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "0bfcdf28c056408a9ff6c748d7d7f3b4"
format_stamp: "Formatted at 2026-06-20 14:53:32 on dist-test-slave-sms9"
server_key: "5d310201ea54792618c941673a375f43"
server_key_iv: "1a49d79106be694e2dde1127cf0b7193"
server_key_version: "encryptionkey@0"
I20260620 14:53:32.337293 14770 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:53:32.339779 14785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:32.341246 14770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 14:53:32.341368 14770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "0bfcdf28c056408a9ff6c748d7d7f3b4"
format_stamp: "Formatted at 2026-06-20 14:53:32 on dist-test-slave-sms9"
server_key: "5d310201ea54792618c941673a375f43"
server_key_iv: "1a49d79106be694e2dde1127cf0b7193"
server_key_version: "encryptionkey@0"
I20260620 14:53:32.341481 14770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:32.366135 14770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:32.369305 14770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:32.369452 14770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:32.377674 14770 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:38593
I20260620 14:53:32.377755 14847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:38593 every 8 connection(s)
I20260620 14:53:32.378625 14770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:32.381345 14848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:32.386699 14848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4: Bootstrap starting.
I20260620 14:53:32.387598 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 14770
I20260620 14:53:32.387746 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:32.388072 7267 external_mini_cluster.cc:1428] Setting key 771b282bc07e530c32e36b4d101d7569
I20260620 14:53:32.389495 14848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:32.390229 14848 log.cc:824] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:32.392026 14848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4: No bootstrap required, opened a new log
I20260620 14:53:32.395373 14848 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfcdf28c056408a9ff6c748d7d7f3b4" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38593 } }
I20260620 14:53:32.395608 14848 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:32.395670 14848 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bfcdf28c056408a9ff6c748d7d7f3b4, State: Initialized, Role: FOLLOWER
I20260620 14:53:32.396101 14848 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfcdf28c056408a9ff6c748d7d7f3b4" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38593 } }
I20260620 14:53:32.396257 14848 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:32.396339 14848 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:32.396457 14848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [term 0 FOLLOWER]: Advancing to term 1
Jun 20 14:53:32 dist-test-slave-sms9 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:32.397504 14848 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfcdf28c056408a9ff6c748d7d7f3b4" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38593 } }
I20260620 14:53:32.397847 14848 leader_election.cc:302] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bfcdf28c056408a9ff6c748d7d7f3b4; no voters:
I20260620 14:53:32.398208 14848 leader_election.cc:288] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:32.398360 14853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:32.398597 14853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [term 1 LEADER]: Becoming Leader. State: Replica: 0bfcdf28c056408a9ff6c748d7d7f3b4, State: Running, Role: LEADER
I20260620 14:53:32.398937 14853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfcdf28c056408a9ff6c748d7d7f3b4" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38593 } }
I20260620 14:53:32.399477 14848 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:32.400166 14853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bfcdf28c056408a9ff6c748d7d7f3b4. Latest consensus state: current_term: 1 leader_uuid: "0bfcdf28c056408a9ff6c748d7d7f3b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfcdf28c056408a9ff6c748d7d7f3b4" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38593 } } }
I20260620 14:53:32.400285 14853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:32.400628 14854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bfcdf28c056408a9ff6c748d7d7f3b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bfcdf28c056408a9ff6c748d7d7f3b4" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38593 } } }
I20260620 14:53:32.400753 14854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:32.400648 14860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:32.403332 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:32.389173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46612 (local address 127.7.24.254:38593)
0620 14:53:32.389648 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:32.389664 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:32.389703 (+ 39us) server_negotiation.cc:407] Connection header received
0620 14:53:32.390326 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:32.390350 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:32.390737 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:32.391053 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:32.391852 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.392856 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.393729 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.394203 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.394374 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:32.397185 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:32.397191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:32.397206 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:32.397232 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:32.399366 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:32.399867 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:32.399873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:32.399878 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:32.400024 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:32.400481 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:32.400485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:32.400487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:32.401113 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:32.401928 (+ 815us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:32.402140 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 14:53:32.402423 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":149,"threads_started":1}
I20260620 14:53:32.403849 14860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:32.408980 14860 catalog_manager.cc:1347] Generated new cluster ID: bdf8be2a07144750ba544764c9445db5
I20260620 14:53:32.409065 14860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:32.424172 14860 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:32.425659 14860 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:32.434391 14860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4: Generated new TSK 0
I20260620 14:53:32.435132 14860 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab.
Entry for principal oryx/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab.
I20260620 14:53:32.471654 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:38593
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab
--principal=oryx/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:32.583891 14874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:32.584141 14874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:32.584190 14874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:32.588127 14874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:32.588183 14874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:32.588222 14874 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 14:53:32.588253 14874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:32.593324 14874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab
--principal=oryx/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.24.254:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:32.594278 14874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:32.595224 14874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:32.604792 14879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:32.604739 14882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:32.604815 14880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:32.604871 14874 server_base.cc:1034] running on GCE node
I20260620 14:53:32.605391 14874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:32.606037 14874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:32.607223 14874 hybrid_clock.cc:648] HybridClock initialized: now 1781967212607191 us; error 46 us; skew 500 ppm
Jun 20 14:53:32 dist-test-slave-sms9 krb5kdc[14757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967212, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:32.609942 14874 init.cc:377] Logged in from keytab as oryx/127.7.24.193@KRBTEST.COM (short username oryx)
I20260620 14:53:32.610872 14874 webserver.cc:460] Webserver started at http://127.7.24.193:36003/ using document root <none> and password file <none>
I20260620 14:53:32.611382 14874 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:32.611445 14874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:32.611605 14874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:32.613489 14874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "378c857c8c2741ef92b37e93b197fc91"
format_stamp: "Formatted at 2026-06-20 14:53:32 on dist-test-slave-sms9"
server_key: "6d739f45748f1a200b9ac54d76e465aa"
server_key_iv: "776a4d6c05a086b06622136037473692"
server_key_version: "encryptionkey@0"
I20260620 14:53:32.613976 14874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "378c857c8c2741ef92b37e93b197fc91"
format_stamp: "Formatted at 2026-06-20 14:53:32 on dist-test-slave-sms9"
server_key: "6d739f45748f1a200b9ac54d76e465aa"
server_key_iv: "776a4d6c05a086b06622136037473692"
server_key_version: "encryptionkey@0"
I20260620 14:53:32.617630 14874 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 14:53:32.620210 14889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:32.621673 14874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 14:53:32.621795 14874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "378c857c8c2741ef92b37e93b197fc91"
format_stamp: "Formatted at 2026-06-20 14:53:32 on dist-test-slave-sms9"
server_key: "6d739f45748f1a200b9ac54d76e465aa"
server_key_iv: "776a4d6c05a086b06622136037473692"
server_key_version: "encryptionkey@0"
I20260620 14:53:32.621903 14874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:32.633709 14874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:32.636591 14874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:32.636787 14874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:32.638329 14874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:32.638376 14874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:32.638444 14874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:32.638507 14874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 14:53:32 dist-test-slave-sms9 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967212, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:32.649659 14874 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:41913
I20260620 14:53:32.649718 15004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:41913 every 8 connection(s)
I20260620 14:53:32.650585 14874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:32.653119 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:32.639111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:43565 (local address 127.7.24.254:38593)
0620 14:53:32.639293 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:32.639300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:32.640110 (+ 810us) server_negotiation.cc:407] Connection header received
0620 14:53:32.641357 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:32.641360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:32.641423 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:32.641512 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:32.642787 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.643330 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.644797 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.645236 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.645361 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:32.649277 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:32.649282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:32.649285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:32.649314 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:32.650989 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:32.651614 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:32.651618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:32.651619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:32.651710 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:32.652005 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:32.652007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:32.652008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:32.652208 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:32.652286 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:32.652871 (+ 585us) server_negotiation.cc:299] Negotiation successful
0620 14:53:32.652977 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 14:53:32.653878 14898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:32.639455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38593 (local address 127.7.24.193:43565)
0620 14:53:32.639931 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:32.639988 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:32.641110 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:32.641802 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:32.641811 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:32.642144 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:32.642645 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:32.642658 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.643455 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.643458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:32.644679 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:32.644690 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.645410 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.645413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:32.645643 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:32.646261 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:32.646279 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:32.649023 (+ 2744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:32.651134 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:32.651139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:32.651153 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:32.651492 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:32.651797 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:32.651799 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:32.651800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:32.651922 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:32.652353 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:32.652358 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:32.652637 (+ 279us) client_negotiation.cc:769] Sending connection context
0620 14:53:32.652807 (+ 170us) client_negotiation.cc:240] Negotiation successful
0620 14:53:32.653044 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":111,"threads_started":1}
I20260620 14:53:32.655213 15005 heartbeater.cc:347] Connected to a master server at 127.7.24.254:38593
I20260620 14:53:32.655355 15005 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:32.655946 15005 heartbeater.cc:511] Master 127.7.24.254:38593 requested a full tablet report, sending...
I20260620 14:53:32.657512 14802 ts_manager.cc:194] Registered new tserver with Master: 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913)
I20260620 14:53:32.659052 14802 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:43565
I20260620 14:53:32.659123 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 14874
I20260620 14:53:32.659265 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:32.659590 7267 external_mini_cluster.cc:1428] Setting key 4759b56f5ea5300a21b0ef675cce4f80
I20260620 14:53:32.665771 15011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:32.657553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38593 (local address 127.7.24.193:46005)
0620 14:53:32.657944 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:32.657958 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:32.658062 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:32.658309 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:32.658312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:32.658534 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:32.658647 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:32.658654 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.659592 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.659597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:32.660419 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:32.660428 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.660971 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.660974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:32.661077 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:32.661652 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:32.661664 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:32.661990 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:32.664060 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:32.664063 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:32.664065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:32.664262 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:32.664619 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:32.664621 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:32.664623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:32.664707 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:32.665272 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:32.665275 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:32.665386 (+ 111us) client_negotiation.cc:769] Sending connection context
0620 14:53:32.665491 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 14:53:32.665621 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":145,"threads_started":1}
I20260620 14:53:32.665887 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:32.657677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:46005 (local address 127.7.24.254:38593)
0620 14:53:32.657798 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:32.657802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:32.658042 (+ 240us) server_negotiation.cc:407] Connection header received
0620 14:53:32.658149 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:32.658152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:32.658197 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:32.658280 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:32.658761 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.659454 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.660543 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.660860 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.660969 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:32.662143 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:32.662147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:32.662149 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:32.662174 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:32.663933 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:32.664359 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:32.664365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:32.664370 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:32.664461 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:32.664836 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:32.664842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:32.664847 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:32.665113 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:32.665282 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:32.665585 (+ 303us) server_negotiation.cc:299] Negotiation successful
0620 14:53:32.665719 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for oryx/127.7.24.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab.
Entry for principal oryx/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab.
I20260620 14:53:32.690287 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:38593
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab
--principal=oryx/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:32.801285 15014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:32.801604 15014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:32.801689 15014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:32.805896 15014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:32.805977 15014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:32.806046 15014 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 14:53:32.806126 15014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:32.811590 15014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab
--principal=oryx/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.24.254:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:32.812674 15014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:32.813694 15014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:32.823287 15020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:32.823336 15022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:32.823304 15019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:32.823725 15014 server_base.cc:1034] running on GCE node
I20260620 14:53:32.824141 15014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:32.824818 15014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:32.826005 15014 hybrid_clock.cc:648] HybridClock initialized: now 1781967212825986 us; error 31 us; skew 500 ppm
Jun 20 14:53:32 dist-test-slave-sms9 krb5kdc[14757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967212, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:32.828851 15014 init.cc:377] Logged in from keytab as oryx/127.7.24.194@KRBTEST.COM (short username oryx)
I20260620 14:53:32.829734 15014 webserver.cc:460] Webserver started at http://127.7.24.194:33335/ using document root <none> and password file <none>
I20260620 14:53:32.830271 15014 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:32.830328 15014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:32.830564 15014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:32.832753 15014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "f6946442599346978805099426058f15"
format_stamp: "Formatted at 2026-06-20 14:53:32 on dist-test-slave-sms9"
server_key: "cc9aa01c17f5e6de2992a82958ac3af8"
server_key_iv: "ee8623ff3fcfbdaa1c1d49eaec70bdc3"
server_key_version: "encryptionkey@0"
I20260620 14:53:32.833356 15014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "f6946442599346978805099426058f15"
format_stamp: "Formatted at 2026-06-20 14:53:32 on dist-test-slave-sms9"
server_key: "cc9aa01c17f5e6de2992a82958ac3af8"
server_key_iv: "ee8623ff3fcfbdaa1c1d49eaec70bdc3"
server_key_version: "encryptionkey@0"
I20260620 14:53:32.837319 15014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:53:32.840056 15029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:32.841435 15014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 14:53:32.841545 15014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "f6946442599346978805099426058f15"
format_stamp: "Formatted at 2026-06-20 14:53:32 on dist-test-slave-sms9"
server_key: "cc9aa01c17f5e6de2992a82958ac3af8"
server_key_iv: "ee8623ff3fcfbdaa1c1d49eaec70bdc3"
server_key_version: "encryptionkey@0"
I20260620 14:53:32.841641 15014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:32.852919 15014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:32.855983 15014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:32.856158 15014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:32.857472 15014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:32.857522 15014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:32.857589 15014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:32.857620 15014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 14:53:32 dist-test-slave-sms9 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967212, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:32.868500 15014 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:46159
I20260620 14:53:32.868582 15144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:46159 every 8 connection(s)
I20260620 14:53:32.869649 15014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:32.871543 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:32.858353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:58483 (local address 127.7.24.254:38593)
0620 14:53:32.858534 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:32.858539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:32.859608 (+ 1069us) server_negotiation.cc:407] Connection header received
0620 14:53:32.860844 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:32.860851 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:32.860926 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:32.861036 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:32.862624 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.863164 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.864336 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.864643 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.864763 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:32.867693 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:32.867696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:32.867698 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:32.867722 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:32.869497 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:32.870129 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:32.870133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:32.870135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:32.870226 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:32.870611 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:32.870615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:32.870617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:32.870799 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:32.870879 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:32.871327 (+ 448us) server_negotiation.cc:299] Negotiation successful
0620 14:53:32.871428 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 14:53:32.872315 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:32.858959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38593 (local address 127.7.24.194:58483)
0620 14:53:32.859440 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:32.859484 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:32.860545 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:32.861253 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:32.861263 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:32.861713 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:32.862417 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:32.862437 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.863296 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.863299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:32.864225 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:32.864237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.864815 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.864819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:32.864989 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:32.865601 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:32.865629 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:32.867506 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:32.869627 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:32.869633 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:32.869648 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:32.869981 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:32.870316 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:32.870319 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:32.870321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:32.870520 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:32.870896 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:32.870901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:32.871123 (+ 222us) client_negotiation.cc:769] Sending connection context
0620 14:53:32.871266 (+ 143us) client_negotiation.cc:240] Negotiation successful
0620 14:53:32.871497 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":140,"threads_started":1}
I20260620 14:53:32.873364 15145 heartbeater.cc:347] Connected to a master server at 127.7.24.254:38593
I20260620 14:53:32.873513 15145 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:32.873978 15145 heartbeater.cc:511] Master 127.7.24.254:38593 requested a full tablet report, sending...
I20260620 14:53:32.875010 14801 ts_manager.cc:194] Registered new tserver with Master: f6946442599346978805099426058f15 (127.7.24.194:46159)
I20260620 14:53:32.875746 14801 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:58483
I20260620 14:53:32.876927 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 15014
I20260620 14:53:32.877053 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:32.877373 7267 external_mini_cluster.cc:1428] Setting key e6b08a363ddfccf403b88203728610d2
I20260620 14:53:32.883589 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:32.875696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:33317 (local address 127.7.24.254:38593)
0620 14:53:32.875833 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:32.875837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:32.876099 (+ 262us) server_negotiation.cc:407] Connection header received
0620 14:53:32.876232 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:32.876236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:32.876277 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:32.876366 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:32.876867 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.877507 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.878652 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.878974 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.879082 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:32.880103 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:32.880109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:32.880114 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:32.880144 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:32.881782 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:32.882286 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:32.882289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:32.882290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:32.882363 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:32.882719 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:32.882722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:32.882723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:32.882929 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:32.883012 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:32.883377 (+ 365us) server_negotiation.cc:299] Negotiation successful
0620 14:53:32.883477 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 14:53:32.883958 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:32.875678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38593 (local address 127.7.24.194:33317)
0620 14:53:32.876000 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:32.876027 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:32.876138 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:32.876380 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:32.876383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:32.876585 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:32.876756 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:32.876762 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.877639 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.877642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:32.878522 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:32.878532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:32.879106 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:32.879109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:32.879175 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:32.879657 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:32.879668 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:32.879953 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:32.881898 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:32.881905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:32.881910 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:32.882180 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:32.882488 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:32.882494 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:32.882499 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:32.882613 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:32.883040 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:32.883046 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:32.883174 (+ 128us) client_negotiation.cc:769] Sending connection context
0620 14:53:32.883305 (+ 131us) client_negotiation.cc:240] Negotiation successful
0620 14:53:32.883489 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":66,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for oryx/127.7.24.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab.
Entry for principal oryx/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab.
I20260620 14:53:32.905826 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:38593
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab
--principal=oryx/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:33.015935 15154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:33.016211 15154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:33.016255 15154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:33.020233 15154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:33.020290 15154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:33.020329 15154 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 14:53:33.020363 15154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:33.025739 15154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab
--principal=oryx/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.24.254:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:33.026875 15154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:33.027843 15154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:33.037389 15162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:33.037428 15159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:33.037536 15154 server_base.cc:1034] running on GCE node
W20260620 14:53:33.037407 15160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:33.038003 15154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:33.038746 15154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:33.039937 15154 hybrid_clock.cc:648] HybridClock initialized: now 1781967213039902 us; error 44 us; skew 500 ppm
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[14757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967213, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:33.042750 15154 init.cc:377] Logged in from keytab as oryx/127.7.24.195@KRBTEST.COM (short username oryx)
I20260620 14:53:33.043591 15154 webserver.cc:460] Webserver started at http://127.7.24.195:40115/ using document root <none> and password file <none>
I20260620 14:53:33.044139 15154 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:33.044202 15154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:33.044369 15154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:33.046535 15154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "f1ec9b72626a41299cf689b060de481b"
format_stamp: "Formatted at 2026-06-20 14:53:33 on dist-test-slave-sms9"
server_key: "b6afe1abb5f7ba743c2e8e69cfe3dd3b"
server_key_iv: "3789690520840b4bc0e394a09a387f83"
server_key_version: "encryptionkey@0"
I20260620 14:53:33.047055 15154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "f1ec9b72626a41299cf689b060de481b"
format_stamp: "Formatted at 2026-06-20 14:53:33 on dist-test-slave-sms9"
server_key: "b6afe1abb5f7ba743c2e8e69cfe3dd3b"
server_key_iv: "3789690520840b4bc0e394a09a387f83"
server_key_version: "encryptionkey@0"
I20260620 14:53:33.050608 15154 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 14:53:33.052979 15169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:33.054203 15154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:33.054296 15154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "f1ec9b72626a41299cf689b060de481b"
format_stamp: "Formatted at 2026-06-20 14:53:33 on dist-test-slave-sms9"
server_key: "b6afe1abb5f7ba743c2e8e69cfe3dd3b"
server_key_iv: "3789690520840b4bc0e394a09a387f83"
server_key_version: "encryptionkey@0"
I20260620 14:53:33.054378 15154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:33.079681 15154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:33.082746 15154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:33.082906 15154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:33.084342 15154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:33.084396 15154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:33.084437 15154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:33.084452 15154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967213, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:33.095567 15154 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:34005
I20260620 14:53:33.095611 15284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:34005 every 8 connection(s)
I20260620 14:53:33.096418 15154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:33.097847 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.085142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:57911 (local address 127.7.24.254:38593)
0620 14:53:33.085288 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.085293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.086145 (+ 852us) server_negotiation.cc:407] Connection header received
0620 14:53:33.087335 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.087341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.087404 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.087480 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:33.088860 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.089536 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.090732 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.090998 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.091126 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.094013 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:33.094016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:33.094019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:33.094050 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:33.095800 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:33.096285 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:33.096290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:33.096292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:33.096376 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:33.096750 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:33.096755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:33.096757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:33.096989 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:33.097089 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.097585 (+ 496us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.097721 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 14:53:33.098662 15178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.085539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38593 (local address 127.7.24.195:57911)
0620 14:53:33.085971 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.086019 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.087074 (+ 1055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.087820 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.087826 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.088150 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:33.088705 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.088742 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.089664 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.089668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.090604 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.090620 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.091172 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.091176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.091385 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.092013 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:33.092036 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:33.093848 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:33.095910 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:33.095915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:33.095928 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:33.096180 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:33.096476 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:33.096479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:33.096481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:33.096655 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:33.097087 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:33.097093 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:33.097359 (+ 266us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.097532 (+ 173us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.097771 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":123,"threads_started":1}
I20260620 14:53:33.099803 15285 heartbeater.cc:347] Connected to a master server at 127.7.24.254:38593
I20260620 14:53:33.099956 15285 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:33.100427 15285 heartbeater.cc:511] Master 127.7.24.254:38593 requested a full tablet report, sending...
I20260620 14:53:33.101462 14802 ts_manager.cc:194] Registered new tserver with Master: f1ec9b72626a41299cf689b060de481b (127.7.24.195:34005)
I20260620 14:53:33.102177 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 15154
I20260620 14:53:33.102320 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:33.102283 14802 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:57911
I20260620 14:53:33.102710 7267 external_mini_cluster.cc:1428] Setting key 9c85cb819fdd905e1604a443e5c9f711
I20260620 14:53:33.104677 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:53:33.111129 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.102038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:33447 (local address 127.7.24.254:38593)
0620 14:53:33.102155 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.102159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.102387 (+ 228us) server_negotiation.cc:407] Connection header received
0620 14:53:33.102622 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.102626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.102667 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.102717 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:33.104974 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.105709 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.106496 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.106865 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.106980 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.108049 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:33.108053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:33.108055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:33.108081 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:33.109522 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:33.109941 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:33.109946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:33.109948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:33.110022 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:33.110300 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:33.110304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:33.110306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:33.110512 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:33.110624 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.110868 (+ 244us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.110970 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:53:33.111166 15291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.101954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38593 (local address 127.7.24.195:33447)
0620 14:53:33.102284 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.102302 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.102503 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.104501 (+ 1998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.104504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.104749 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:33.104868 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.104874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.105836 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.105838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.106366 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.106373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.106984 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.106987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.107085 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.107604 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:33.107616 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:33.107919 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:33.109627 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:33.109630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:33.109632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:33.109842 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:33.110121 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:33.110124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:33.110127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:33.110210 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:33.110608 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:33.110611 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:33.110701 (+ 90us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.110816 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.110963 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":89,"threads_started":1}
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[14757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967213, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
W20260620 14:53:33.128656 14851 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:33.122728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46626 (local address 127.7.24.254:38593)
0620 14:53:33.122839 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.122842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.122854 (+ 12us) server_negotiation.cc:407] Connection header received
0620 14:53:33.122890 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.122892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.122927 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.122998 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:33.123711 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.124233 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.125019 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.125224 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.125299 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.128455 (+ 3156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46626: BlockingRecv error: recv got EOF from 127.0.0.1:46626 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967213, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:33.137871 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.130046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46628 (local address 127.7.24.254:38593)
0620 14:53:33.130184 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.130190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.130255 (+ 65us) server_negotiation.cc:407] Connection header received
0620 14:53:33.130389 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.130392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.130436 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.130562 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:33.131190 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.131682 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.132452 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.132668 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.132778 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.135016 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:33.135019 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:33.135021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:33.135042 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:33.136339 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:33.136732 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:33.136735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:33.136736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:33.136817 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:33.137074 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:33.137076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:33.137077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:33.137245 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:33.137350 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.137563 (+ 213us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.137679 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 14:53:33.140930 14802 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:33.144025 14802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:33.159336 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.152694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:34005 (local address 127.0.0.1:54540)
0620 14:53:33.153489 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.153544 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.153678 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.154498 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.154504 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.154536 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:33.154792 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.154805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.156006 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.156013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.157475 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.157486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.158740 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.158743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.158850 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.158890 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.159017 (+ 127us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.159166 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"mutex_wait_us":143,"spinlock_wait_cycles":10496,"thread_start_us":106,"threads_started":1}
I20260620 14:53:33.159473 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.152694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:46159 (local address 127.0.0.1:54594)
0620 14:53:33.153489 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.153544 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.153678 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.154508 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.154510 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.154537 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:33.154958 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.154967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.156429 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.156435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.157582 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.157591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.158940 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.158944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.159067 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.159103 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.159199 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.159273 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":224,"spinlock_wait_cycles":14080,"thread_start_us":62,"threads_started":1}
I20260620 14:53:33.159696 15312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.153451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54594 (local address 127.7.24.194:46159)
0620 14:53:33.153974 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.153986 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.154009 (+ 23us) server_negotiation.cc:407] Connection header received
0620 14:53:33.154084 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.154089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.154296 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.154450 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:33.155076 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.156307 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.157699 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.158590 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.158705 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.158877 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.159235 (+ 358us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.159334 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":93,"threads_started":1}
I20260620 14:53:33.160004 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.152799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54540 (local address 127.7.24.195:34005)
0620 14:53:33.153973 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.153986 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.154009 (+ 23us) server_negotiation.cc:407] Connection header received
0620 14:53:33.154083 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.154089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.154296 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.154755 (+ 459us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:33.154939 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.155891 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.157593 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.158590 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.158715 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.158877 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.159740 (+ 863us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.159863 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1075,"thread_start_us":133,"threads_started":1}
I20260620 14:53:33.161136 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.152991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:41913 (local address 127.0.0.1:57442)
0620 14:53:33.153992 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.154006 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.154158 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.154820 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.154823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.154835 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:33.154976 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.154983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.157691 (+ 2708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.157692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.158572 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.158582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.160717 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.160721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.160808 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.160832 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.160948 (+ 116us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.161017 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"mutex_wait_us":35,"thread_start_us":96,"threads_started":1}
I20260620 14:53:33.161402 15311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.153356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.7.24.193:41913)
0620 14:53:33.153847 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.153862 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.154080 (+ 218us) server_negotiation.cc:407] Connection header received
0620 14:53:33.154309 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.154314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.154528 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.154656 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:33.155162 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.156300 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.158693 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.159734 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.159907 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.160043 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.160990 (+ 947us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.161089 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":79,"threads_started":1}
I20260620 14:53:33.162636 15219 tablet_service.cc:1467] Processing CreateTablet for tablet ee314968307d455987b9cfeede112c48 (DEFAULT_TABLE table=test-table [id=ebbe1f907ad64997bfa94f2802678f65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.162635 15079 tablet_service.cc:1467] Processing CreateTablet for tablet ee314968307d455987b9cfeede112c48 (DEFAULT_TABLE table=test-table [id=ebbe1f907ad64997bfa94f2802678f65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.163590 15079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee314968307d455987b9cfeede112c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.163590 15219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee314968307d455987b9cfeede112c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.170346 15313 tablet_bootstrap.cc:492] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15: Bootstrap starting.
I20260620 14:53:33.171584 15314 tablet_bootstrap.cc:492] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b: Bootstrap starting.
I20260620 14:53:33.171849 14939 tablet_service.cc:1467] Processing CreateTablet for tablet ee314968307d455987b9cfeede112c48 (DEFAULT_TABLE table=test-table [id=ebbe1f907ad64997bfa94f2802678f65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.172677 15313 tablet_bootstrap.cc:654] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.172700 14939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee314968307d455987b9cfeede112c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.173488 15314 tablet_bootstrap.cc:654] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.173519 15313 log.cc:824] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:33.174254 15314 log.cc:824] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:33.175714 15313 tablet_bootstrap.cc:492] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15: No bootstrap required, opened a new log
I20260620 14:53:33.175907 15313 ts_tablet_manager.cc:1397] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:53:33.179104 15317 tablet_bootstrap.cc:492] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91: Bootstrap starting.
I20260620 14:53:33.179006 15313 raft_consensus.cc:348] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } }
I20260620 14:53:33.179281 15313 raft_consensus.cc:374] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.179333 15313 raft_consensus.cc:729] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6946442599346978805099426058f15, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.179806 15313 consensus_queue.cc:260] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } }
I20260620 14:53:33.180258 15314 tablet_bootstrap.cc:492] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b: No bootstrap required, opened a new log
I20260620 14:53:33.180444 15314 ts_tablet_manager.cc:1397] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260620 14:53:33.180473 15145 heartbeater.cc:503] Master 127.7.24.254:38593 was elected leader, sending a full tablet report...
I20260620 14:53:33.181871 15313 ts_tablet_manager.cc:1428] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 14:53:33.181650 15317 tablet_bootstrap.cc:654] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.183133 15317 log.cc:824] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:33.184273 15314 raft_consensus.cc:348] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } }
I20260620 14:53:33.184545 15314 raft_consensus.cc:374] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.184608 15314 raft_consensus.cc:729] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ec9b72626a41299cf689b060de481b, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.185081 15314 consensus_queue.cc:260] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } }
I20260620 14:53:33.185197 15317 tablet_bootstrap.cc:492] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91: No bootstrap required, opened a new log
I20260620 14:53:33.185395 15317 ts_tablet_manager.cc:1397] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260620 14:53:33.185801 15285 heartbeater.cc:503] Master 127.7.24.254:38593 was elected leader, sending a full tablet report...
I20260620 14:53:33.186061 15314 ts_tablet_manager.cc:1428] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260620 14:53:33.189751 15317 raft_consensus.cc:348] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } }
I20260620 14:53:33.189975 15317 raft_consensus.cc:374] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.190040 15317 raft_consensus.cc:729] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378c857c8c2741ef92b37e93b197fc91, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.190539 15317 consensus_queue.cc:260] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } }
I20260620 14:53:33.191146 15005 heartbeater.cc:503] Master 127.7.24.254:38593 was elected leader, sending a full tablet report...
I20260620 14:53:33.191360 15317 ts_tablet_manager.cc:1428] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260620 14:53:33.191565 15320 raft_consensus.cc:482] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:33.191730 15320 raft_consensus.cc:504] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } }
I20260620 14:53:33.193487 15320 leader_election.cc:288] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6946442599346978805099426058f15 (127.7.24.194:46159), 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913)
I20260620 14:53:33.197257 15312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.193294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:41741 (local address 127.7.24.194:46159)
0620 14:53:33.193403 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.193415 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.193444 (+ 29us) server_negotiation.cc:407] Connection header received
0620 14:53:33.193493 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.193495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.193549 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.193627 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:33.194025 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.194766 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.196315 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.196703 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.196834 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.196887 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.197064 (+ 177us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.197134 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 14:53:33.197269 15178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.193019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:46159 (local address 127.7.24.195:41741)
0620 14:53:33.193175 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.193193 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.193277 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.193747 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.193750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.193767 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:33.193879 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.193886 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.194871 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.194875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.196196 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.196206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.196824 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.196828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.196932 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.196954 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.197051 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.197142 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260620 14:53:33.197701 15311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.193431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:43039 (local address 127.7.24.193:41913)
0620 14:53:33.193644 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.193648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.193861 (+ 213us) server_negotiation.cc:407] Connection header received
0620 14:53:33.193978 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.193981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.194020 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.194110 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:33.194385 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.195010 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.196724 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.197135 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.197265 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.197310 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.197530 (+ 220us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.197605 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260620 14:53:33.197930 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee314968307d455987b9cfeede112c48" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6946442599346978805099426058f15" is_pre_election: true
I20260620 14:53:33.198017 15322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.193054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:41913 (local address 127.7.24.195:43039)
0620 14:53:33.193765 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.193784 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.193898 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.194120 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.194122 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.194137 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:33.194260 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.194267 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.195151 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.195154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.196605 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.196612 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.197257 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.197268 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.197367 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.197415 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.197845 (+ 430us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.197920 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":125,"threads_started":1}
I20260620 14:53:33.198241 15099 raft_consensus.cc:2454] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 0.
I20260620 14:53:33.198607 14959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee314968307d455987b9cfeede112c48" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378c857c8c2741ef92b37e93b197fc91" is_pre_election: true
I20260620 14:53:33.198776 15171 leader_election.cc:302] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f1ec9b72626a41299cf689b060de481b, f6946442599346978805099426058f15; no voters:
I20260620 14:53:33.198849 14959 raft_consensus.cc:2454] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 0.
I20260620 14:53:33.199043 15320 raft_consensus.cc:2781] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:33.199137 15320 raft_consensus.cc:482] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:33.199231 15320 raft_consensus.cc:3037] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.200438 15320 raft_consensus.cc:504] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } }
I20260620 14:53:33.200815 15320 leader_election.cc:288] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 election: Requested vote from peers f6946442599346978805099426058f15 (127.7.24.194:46159), 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913)
I20260620 14:53:33.201186 14959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee314968307d455987b9cfeede112c48" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378c857c8c2741ef92b37e93b197fc91"
I20260620 14:53:33.201377 14959 raft_consensus.cc:3037] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.202447 14959 raft_consensus.cc:2454] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 1.
I20260620 14:53:33.202651 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee314968307d455987b9cfeede112c48" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6946442599346978805099426058f15"
I20260620 14:53:33.202821 15099 raft_consensus.cc:3037] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.202885 15170 leader_election.cc:302] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378c857c8c2741ef92b37e93b197fc91, f1ec9b72626a41299cf689b060de481b; no voters:
I20260620 14:53:33.203075 15320 raft_consensus.cc:2781] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:33.203320 15320 raft_consensus.cc:686] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 1 LEADER]: Becoming Leader. State: Replica: f1ec9b72626a41299cf689b060de481b, State: Running, Role: LEADER
I20260620 14:53:33.203693 15320 consensus_queue.cc:237] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } }
I20260620 14:53:33.204156 15099 raft_consensus.cc:2454] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 1.
I20260620 14:53:33.206730 14802 catalog_manager.cc:5495] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b reported cstate change: term changed from 0 to 1, leader changed from <none> to f1ec9b72626a41299cf689b060de481b (127.7.24.195). New cstate: current_term: 1 leader_uuid: "f1ec9b72626a41299cf689b060de481b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } health_report { overall_health: UNKNOWN } } }
I20260620 14:53:33.221925 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.218122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54554 (local address 127.7.24.195:34005)
0620 14:53:33.218267 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.218271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.218286 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:53:33.218367 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.218370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.218427 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.218593 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:33.218831 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.219401 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.220408 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.220655 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.220745 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.221033 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:33.221484 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:33.221591 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.221760 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.221818 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 14:53:33.227878 14959 raft_consensus.cc:1261] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Refusing update from remote peer f1ec9b72626a41299cf689b060de481b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:33.228149 15099 raft_consensus.cc:1261] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Refusing update from remote peer f1ec9b72626a41299cf689b060de481b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:33.228730 15323 consensus_queue.cc:1035] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [LEADER]: Connected to new peer: Peer: permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:33.228919 15320 consensus_queue.cc:1035] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:33.237242 15328 mvcc.cc:205] Tried to move back new op lower bound from 7298937705373753344 to 7298937705285517312. Current Snapshot: MvccSnapshot[applied={T|T < 7298937705373753344}]
I20260620 14:53:33.245093 14802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:33.245290 14802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:33.247893 14802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4: Sending DeleteTablet for 3 replicas of tablet ee314968307d455987b9cfeede112c48
I20260620 14:53:33.248745 15079 tablet_service.cc:1514] Processing DeleteTablet for tablet ee314968307d455987b9cfeede112c48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54594
I20260620 14:53:33.248739 15219 tablet_service.cc:1514] Processing DeleteTablet for tablet ee314968307d455987b9cfeede112c48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54540
I20260620 14:53:33.249274 15336 tablet_replica.cc:330] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b: stopping tablet replica
I20260620 14:53:33.249648 15336 raft_consensus.cc:2229] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:33.249651 14939 tablet_service.cc:1514] Processing DeleteTablet for tablet ee314968307d455987b9cfeede112c48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:57442
I20260620 14:53:33.250144 15338 tablet_replica.cc:330] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91: stopping tablet replica
I20260620 14:53:33.250160 15336 raft_consensus.cc:2258] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:33.250308 14802 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 14:53:33.250422 15338 raft_consensus.cc:2229] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:33.250747 15338 raft_consensus.cc:2258] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 14:53:33.250774 14802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:33.250928 15337 tablet_replica.cc:330] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15: stopping tablet replica
I20260620 14:53:33.251282 15337 raft_consensus.cc:2229] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:33.251649 15337 raft_consensus.cc:2258] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:33.252097 15336 ts_tablet_manager.cc:1905] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:33.252108 15338 ts_tablet_manager.cc:1905] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:33.253046 15337 ts_tablet_manager.cc:1905] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:33.254868 15338 ts_tablet_manager.cc:1918] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:33.254978 15338 log.cc:1196] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/wal/wals/ee314968307d455987b9cfeede112c48
I20260620 14:53:33.255309 15338 ts_tablet_manager.cc:1939] T ee314968307d455987b9cfeede112c48 P 378c857c8c2741ef92b37e93b197fc91: Deleting consensus metadata
I20260620 14:53:33.255990 15336 ts_tablet_manager.cc:1918] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:33.255985 15337 ts_tablet_manager.cc:1918] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:33.256086 15336 log.cc:1196] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/ee314968307d455987b9cfeede112c48
I20260620 14:53:33.256238 15337 log.cc:1196] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/ee314968307d455987b9cfeede112c48
I20260620 14:53:33.256455 15336 ts_tablet_manager.cc:1939] T ee314968307d455987b9cfeede112c48 P f1ec9b72626a41299cf689b060de481b: Deleting consensus metadata
I20260620 14:53:33.256474 15219 tablet_service.cc:1467] Processing CreateTablet for tablet e9481e4fac514e92b63d3521266d46f7 (DEFAULT_TABLE table=test-table [id=c5bb95a233004684a9a2cc69262c5212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.256742 15337 ts_tablet_manager.cc:1939] T ee314968307d455987b9cfeede112c48 P f6946442599346978805099426058f15: Deleting consensus metadata
I20260620 14:53:33.256776 15219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9481e4fac514e92b63d3521266d46f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.257521 14788 catalog_manager.cc:4841] TS f1ec9b72626a41299cf689b060de481b (127.7.24.195:34005): tablet ee314968307d455987b9cfeede112c48 (table test-table [id=ebbe1f907ad64997bfa94f2802678f65]) successfully deleted
I20260620 14:53:33.258701 14786 catalog_manager.cc:4841] TS 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913): tablet ee314968307d455987b9cfeede112c48 (table test-table [id=ebbe1f907ad64997bfa94f2802678f65]) successfully deleted
I20260620 14:53:33.258965 15314 tablet_bootstrap.cc:492] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b: Bootstrap starting.
I20260620 14:53:33.258963 14787 catalog_manager.cc:4841] TS f6946442599346978805099426058f15 (127.7.24.194:46159): tablet ee314968307d455987b9cfeede112c48 (table test-table [id=ebbe1f907ad64997bfa94f2802678f65]) successfully deleted
I20260620 14:53:33.259035 14939 tablet_service.cc:1467] Processing CreateTablet for tablet e9481e4fac514e92b63d3521266d46f7 (DEFAULT_TABLE table=test-table [id=c5bb95a233004684a9a2cc69262c5212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.259262 14939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9481e4fac514e92b63d3521266d46f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.259421 15079 tablet_service.cc:1467] Processing CreateTablet for tablet e9481e4fac514e92b63d3521266d46f7 (DEFAULT_TABLE table=test-table [id=c5bb95a233004684a9a2cc69262c5212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.259654 15079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9481e4fac514e92b63d3521266d46f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.259927 15314 tablet_bootstrap.cc:654] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.261437 15314 tablet_bootstrap.cc:492] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b: No bootstrap required, opened a new log
I20260620 14:53:33.261511 15314 ts_tablet_manager.cc:1397] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:33.261878 15317 tablet_bootstrap.cc:492] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91: Bootstrap starting.
I20260620 14:53:33.261881 15313 tablet_bootstrap.cc:492] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15: Bootstrap starting.
I20260620 14:53:33.261972 15314 raft_consensus.cc:348] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.262069 15314 raft_consensus.cc:374] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.262100 15314 raft_consensus.cc:729] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ec9b72626a41299cf689b060de481b, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.262228 15314 consensus_queue.cc:260] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.262518 15314 ts_tablet_manager.cc:1428] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:33.262842 15313 tablet_bootstrap.cc:654] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.262951 15317 tablet_bootstrap.cc:654] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.264626 15317 tablet_bootstrap.cc:492] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91: No bootstrap required, opened a new log
I20260620 14:53:33.264748 15317 ts_tablet_manager.cc:1397] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:33.265154 15317 raft_consensus.cc:348] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.265270 15317 raft_consensus.cc:374] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.265314 15317 raft_consensus.cc:729] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378c857c8c2741ef92b37e93b197fc91, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.265466 15317 consensus_queue.cc:260] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.265754 15317 ts_tablet_manager.cc:1428] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 14:53:33.266289 15313 tablet_bootstrap.cc:492] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15: No bootstrap required, opened a new log
I20260620 14:53:33.266381 15313 ts_tablet_manager.cc:1397] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260620 14:53:33.266791 15313 raft_consensus.cc:348] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.266906 15313 raft_consensus.cc:374] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.266943 15313 raft_consensus.cc:729] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6946442599346978805099426058f15, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.267076 15313 consensus_queue.cc:260] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.267313 15313 ts_tablet_manager.cc:1428] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:33.289626 15320 raft_consensus.cc:482] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:33.289770 15320 raft_consensus.cc:504] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.290194 15320 leader_election.cc:288] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913), f6946442599346978805099426058f15 (127.7.24.194:46159)
I20260620 14:53:33.290643 14959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9481e4fac514e92b63d3521266d46f7" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378c857c8c2741ef92b37e93b197fc91" is_pre_election: true
I20260620 14:53:33.290712 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9481e4fac514e92b63d3521266d46f7" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6946442599346978805099426058f15" is_pre_election: true
I20260620 14:53:33.290814 14959 raft_consensus.cc:2454] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 0.
I20260620 14:53:33.290871 15099 raft_consensus.cc:2454] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 0.
I20260620 14:53:33.291169 15171 leader_election.cc:302] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f1ec9b72626a41299cf689b060de481b, f6946442599346978805099426058f15; no voters:
I20260620 14:53:33.291416 15320 raft_consensus.cc:2781] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:33.291484 15320 raft_consensus.cc:482] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:33.291543 15320 raft_consensus.cc:3037] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.292506 15320 raft_consensus.cc:504] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.292850 15320 leader_election.cc:288] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 election: Requested vote from peers 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913), f6946442599346978805099426058f15 (127.7.24.194:46159)
I20260620 14:53:33.293195 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9481e4fac514e92b63d3521266d46f7" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6946442599346978805099426058f15"
I20260620 14:53:33.293316 15099 raft_consensus.cc:3037] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.293598 14959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9481e4fac514e92b63d3521266d46f7" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378c857c8c2741ef92b37e93b197fc91"
I20260620 14:53:33.293730 14959 raft_consensus.cc:3037] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.294415 15099 raft_consensus.cc:2454] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 1.
I20260620 14:53:33.294674 14959 raft_consensus.cc:2454] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 1.
I20260620 14:53:33.294716 15171 leader_election.cc:302] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f1ec9b72626a41299cf689b060de481b, f6946442599346978805099426058f15; no voters:
I20260620 14:53:33.294865 15320 raft_consensus.cc:2781] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:33.294989 15320 raft_consensus.cc:686] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 1 LEADER]: Becoming Leader. State: Replica: f1ec9b72626a41299cf689b060de481b, State: Running, Role: LEADER
I20260620 14:53:33.295132 15320 consensus_queue.cc:237] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.296818 14802 catalog_manager.cc:5495] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b reported cstate change: term changed from 0 to 1, leader changed from <none> to f1ec9b72626a41299cf689b060de481b (127.7.24.195). New cstate: current_term: 1 leader_uuid: "f1ec9b72626a41299cf689b060de481b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } health_report { overall_health: HEALTHY } } }
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967212, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.254@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:33.341935 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.332426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38593 (local address 127.0.0.1:46640)
0620 14:53:33.332767 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.332785 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.332906 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.333234 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.333236 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.333637 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:33.333765 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.333771 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.334846 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.334848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.335410 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.335417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.335976 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.335979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.336085 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.336720 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:33.336732 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:33.338299 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:33.340398 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:33.340406 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:33.340411 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:33.340642 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:33.340943 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:33.340946 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:33.340947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:33.341027 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:33.341468 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:33.341474 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:33.341574 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.341677 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.341793 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":32,"thread_start_us":154,"threads_started":1}
I20260620 14:53:33.342057 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.332480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46640 (local address 127.7.24.254:38593)
0620 14:53:33.332623 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.332628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.332869 (+ 241us) server_negotiation.cc:407] Connection header received
0620 14:53:33.333005 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.333009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.333060 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.333168 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:33.333889 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.334720 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.335525 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.335862 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.335961 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.338442 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:33.338446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:33.338448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:33.338503 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:33.340272 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:33.340745 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:33.340749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:33.340751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:33.340844 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:33.341119 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:33.341123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:33.341125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:33.341347 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:33.341446 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.341756 (+ 310us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.341874 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":52}
W20260620 14:53:33.347321 15286 tablet.cc:2369] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:53:33.347249 14802 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:46640:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 14:53:33.348017 14802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:33.353048 15079 tablet_service.cc:1467] Processing CreateTablet for tablet cb7f64f44f264a1e8e33cc395df4586e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5833c11057284506a5b78fc2dbd36a40]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 14:53:33.352993 14939 tablet_service.cc:1467] Processing CreateTablet for tablet cb7f64f44f264a1e8e33cc395df4586e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5833c11057284506a5b78fc2dbd36a40]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 14:53:33.353415 15079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb7f64f44f264a1e8e33cc395df4586e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.353420 14939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb7f64f44f264a1e8e33cc395df4586e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.356271 15219 tablet_service.cc:1467] Processing CreateTablet for tablet cb7f64f44f264a1e8e33cc395df4586e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5833c11057284506a5b78fc2dbd36a40]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 14:53:33.356467 15317 tablet_bootstrap.cc:492] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: Bootstrap starting.
I20260620 14:53:33.356628 15219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb7f64f44f264a1e8e33cc395df4586e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.357493 15317 tablet_bootstrap.cc:654] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.358850 15317 tablet_bootstrap.cc:492] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: No bootstrap required, opened a new log
I20260620 14:53:33.358940 15317 ts_tablet_manager.cc:1397] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:33.359411 15317 raft_consensus.cc:348] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.359503 15314 tablet_bootstrap.cc:492] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: Bootstrap starting.
I20260620 14:53:33.359524 15317 raft_consensus.cc:374] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.359556 15317 raft_consensus.cc:729] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378c857c8c2741ef92b37e93b197fc91, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.359670 15317 consensus_queue.cc:260] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.359746 15313 tablet_bootstrap.cc:492] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: Bootstrap starting.
I20260620 14:53:33.360327 15321 tablet_replica.cc:439] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } }
I20260620 14:53:33.360455 15321 tablet_replica.cc:442] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:33.360460 15314 tablet_bootstrap.cc:654] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.360728 15313 tablet_bootstrap.cc:654] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.361326 15317 ts_tablet_manager.cc:1428] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:33.362249 15313 tablet_bootstrap.cc:492] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: No bootstrap required, opened a new log
I20260620 14:53:33.362347 15313 ts_tablet_manager.cc:1397] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:33.362830 15313 raft_consensus.cc:348] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.362955 15313 raft_consensus.cc:374] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.362998 15313 raft_consensus.cc:729] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6946442599346978805099426058f15, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.363010 15314 tablet_bootstrap.cc:492] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: No bootstrap required, opened a new log
I20260620 14:53:33.363085 15314 ts_tablet_manager.cc:1397] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 14:53:33.363108 15313 consensus_queue.cc:260] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.363390 15313 ts_tablet_manager.cc:1428] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:33.363660 15318 tablet_replica.cc:439] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } }
I20260620 14:53:33.363682 15314 raft_consensus.cc:348] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.363801 15314 raft_consensus.cc:374] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.363823 15318 tablet_replica.cc:442] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:33.363838 15314 raft_consensus.cc:729] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ec9b72626a41299cf689b060de481b, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.363951 15314 consensus_queue.cc:260] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.364717 15314 ts_tablet_manager.cc:1428] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:33.364667 15320 tablet_replica.cc:439] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } }
I20260620 14:53:33.364847 15320 tablet_replica.cc:442] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: This TxnStatusTablet replica's current role is: FOLLOWER
W20260620 14:53:33.370815 15146 tablet.cc:2369] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 14:53:33.401932 15006 tablet.cc:2369] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:53:33.574693 15320 raft_consensus.cc:482] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:33.574836 15320 raft_consensus.cc:504] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.575341 15320 leader_election.cc:288] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913), f6946442599346978805099426058f15 (127.7.24.194:46159)
I20260620 14:53:33.575865 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7f64f44f264a1e8e33cc395df4586e" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6946442599346978805099426058f15" is_pre_election: true
I20260620 14:53:33.575884 14959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7f64f44f264a1e8e33cc395df4586e" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378c857c8c2741ef92b37e93b197fc91" is_pre_election: true
I20260620 14:53:33.576030 15099 raft_consensus.cc:2454] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 0.
I20260620 14:53:33.576051 14959 raft_consensus.cc:2454] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 0.
I20260620 14:53:33.576351 15171 leader_election.cc:302] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f1ec9b72626a41299cf689b060de481b, f6946442599346978805099426058f15; no voters:
I20260620 14:53:33.576601 15320 raft_consensus.cc:2781] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:33.576664 15320 raft_consensus.cc:482] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:33.576692 15320 raft_consensus.cc:3037] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.577692 15320 raft_consensus.cc:504] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.578104 15320 leader_election.cc:288] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 election: Requested vote from peers 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913), f6946442599346978805099426058f15 (127.7.24.194:46159)
I20260620 14:53:33.578446 14959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7f64f44f264a1e8e33cc395df4586e" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378c857c8c2741ef92b37e93b197fc91"
I20260620 14:53:33.578603 14959 raft_consensus.cc:3037] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.578503 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb7f64f44f264a1e8e33cc395df4586e" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6946442599346978805099426058f15"
I20260620 14:53:33.578691 15099 raft_consensus.cc:3037] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.579512 14959 raft_consensus.cc:2454] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 1.
I20260620 14:53:33.579659 15099 raft_consensus.cc:2454] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 1.
I20260620 14:53:33.579805 15170 leader_election.cc:302] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378c857c8c2741ef92b37e93b197fc91, f1ec9b72626a41299cf689b060de481b; no voters:
I20260620 14:53:33.579975 15320 raft_consensus.cc:2781] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:33.580101 15320 raft_consensus.cc:686] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [term 1 LEADER]: Becoming Leader. State: Replica: f1ec9b72626a41299cf689b060de481b, State: Running, Role: LEADER
I20260620 14:53:33.580271 15320 consensus_queue.cc:237] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.580868 15331 tablet_replica.cc:439] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: TxnStatusTablet state changed. Reason: New leader f1ec9b72626a41299cf689b060de481b. Latest consensus state: current_term: 1 leader_uuid: "f1ec9b72626a41299cf689b060de481b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } }
I20260620 14:53:33.581001 15331 tablet_replica.cc:442] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: This TxnStatusTablet replica's current role is: LEADER
I20260620 14:53:33.581245 15353 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 14:53:33.581430 15353 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 14:53:33.582013 14801 catalog_manager.cc:5495] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b reported cstate change: term changed from 0 to 1, leader changed from <none> to f1ec9b72626a41299cf689b060de481b (127.7.24.195). New cstate: current_term: 1 leader_uuid: "f1ec9b72626a41299cf689b060de481b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } health_report { overall_health: HEALTHY } } }
I20260620 14:53:33.605551 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.601742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:34005 (local address 127.0.0.1:54570)
0620 14:53:33.601986 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.602002 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.602102 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.602421 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.602424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.602432 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 14:53:33.602585 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.602592 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.603318 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.603321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.604274 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.604284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.604792 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.604795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.604890 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.604944 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 14:53:33.605237 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 14:53:33.605251 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.605319 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.605404 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260620 14:53:33.605549 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.601862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54570 (local address 127.7.24.195:34005)
0620 14:53:33.602008 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.602014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.602038 (+ 24us) server_negotiation.cc:407] Connection header received
0620 14:53:33.602247 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.602253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.602312 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.602386 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:33.602703 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.603216 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.604421 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.604681 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.604769 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.605084 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:33.605154 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:33.605211 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.605393 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.605444 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:53:33.608269 15099 raft_consensus.cc:1261] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Refusing update from remote peer f1ec9b72626a41299cf689b060de481b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:33.608345 14959 raft_consensus.cc:1261] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Refusing update from remote peer f1ec9b72626a41299cf689b060de481b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:33.608798 15331 consensus_queue.cc:1035] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [LEADER]: Connected to new peer: Peer: permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:33.608945 15320 consensus_queue.cc:1035] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:33.610109 15318 tablet_replica.cc:439] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: TxnStatusTablet state changed. Reason: New leader f1ec9b72626a41299cf689b060de481b. Latest consensus state: current_term: 1 leader_uuid: "f1ec9b72626a41299cf689b060de481b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } }
I20260620 14:53:33.610227 15318 tablet_replica.cc:442] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:33.611150 15321 tablet_replica.cc:439] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: TxnStatusTablet state changed. Reason: New leader f1ec9b72626a41299cf689b060de481b. Latest consensus state: current_term: 1 leader_uuid: "f1ec9b72626a41299cf689b060de481b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } }
I20260620 14:53:33.611254 15321 tablet_replica.cc:442] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:33.611475 15331 tablet_replica.cc:439] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1ec9b72626a41299cf689b060de481b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } }
I20260620 14:53:33.611569 15331 tablet_replica.cc:442] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: This TxnStatusTablet replica's current role is: LEADER
I20260620 14:53:33.612568 15320 tablet_replica.cc:439] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1ec9b72626a41299cf689b060de481b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } }
I20260620 14:53:33.612658 15320 tablet_replica.cc:442] T cb7f64f44f264a1e8e33cc395df4586e P f1ec9b72626a41299cf689b060de481b: This TxnStatusTablet replica's current role is: LEADER
I20260620 14:53:33.613387 15327 mvcc.cc:205] Tried to move back new op lower bound from 7298937706935341056 to 7298937706826080256. Current Snapshot: MvccSnapshot[applied={T|T < 7298937706935341056}]
I20260620 14:53:33.613595 15321 tablet_replica.cc:439] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1ec9b72626a41299cf689b060de481b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } }
I20260620 14:53:33.613678 15321 tablet_replica.cc:442] T cb7f64f44f264a1e8e33cc395df4586e P 378c857c8c2741ef92b37e93b197fc91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:33.614071 15326 mvcc.cc:205] Tried to move back new op lower bound from 7298937706935341056 to 7298937706826080256. Current Snapshot: MvccSnapshot[applied={T|T < 7298937706935341056 or (T in {7298937706935341056})}]
I20260620 14:53:33.615592 15318 tablet_replica.cc:439] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1ec9b72626a41299cf689b060de481b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } } }
I20260620 14:53:33.615705 15318 tablet_replica.cc:442] T cb7f64f44f264a1e8e33cc395df4586e P f6946442599346978805099426058f15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 14:53:33.626636 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.622949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:32805 (local address 127.7.24.195:34005)
0620 14:53:33.623056 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.623060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.623270 (+ 210us) server_negotiation.cc:407] Connection header received
0620 14:53:33.623384 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.623386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.623432 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.623492 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:33.623779 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.624279 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.625456 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.625736 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.625852 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.626163 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:33.626234 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:33.626313 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.626419 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.626480 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":26}
I20260620 14:53:33.626645 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.622874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:34005 (local address 127.7.24.195:32805)
0620 14:53:33.623184 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.623199 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.623300 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.623523 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.623526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.623533 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 14:53:33.623675 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.623681 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.624371 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.624374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.625327 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.625337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.625850 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.625853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.625996 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.626040 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 14:53:33.626335 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 14:53:33.626340 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.626405 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.626507 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":49,"thread_start_us":130,"threads_started":1}
I20260620 14:53:33.632884 15099 raft_consensus.cc:1261] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Refusing update from remote peer f1ec9b72626a41299cf689b060de481b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:33.633278 15323 consensus_queue.cc:1035] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:33.633392 14959 raft_consensus.cc:1261] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Refusing update from remote peer f1ec9b72626a41299cf689b060de481b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:33.633704 15323 consensus_queue.cc:1035] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [LEADER]: Connected to new peer: Peer: permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:33.674567 14801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:33.674713 14801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:33.676826 14801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4: Sending DeleteTablet for 3 replicas of tablet e9481e4fac514e92b63d3521266d46f7
I20260620 14:53:33.677416 14939 tablet_service.cc:1514] Processing DeleteTablet for tablet e9481e4fac514e92b63d3521266d46f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:57442
I20260620 14:53:33.677513 15219 tablet_service.cc:1514] Processing DeleteTablet for tablet e9481e4fac514e92b63d3521266d46f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54540
I20260620 14:53:33.677650 15079 tablet_service.cc:1514] Processing DeleteTablet for tablet e9481e4fac514e92b63d3521266d46f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54594
I20260620 14:53:33.677693 15336 tablet_replica.cc:330] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b: stopping tablet replica
I20260620 14:53:33.677812 15336 raft_consensus.cc:2229] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:33.677832 15338 tablet_replica.cc:330] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91: stopping tablet replica
I20260620 14:53:33.677928 15338 raft_consensus.cc:2229] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:33.678030 15336 raft_consensus.cc:2258] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:33.678058 15338 raft_consensus.cc:2258] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:33.678226 15337 tablet_replica.cc:330] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15: stopping tablet replica
I20260620 14:53:33.678352 15337 raft_consensus.cc:2229] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:33.678499 15337 raft_consensus.cc:2258] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:33.679450 15338 ts_tablet_manager.cc:1905] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:33.681843 15338 ts_tablet_manager.cc:1918] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:33.681942 15338 log.cc:1196] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/wal/wals/e9481e4fac514e92b63d3521266d46f7
I20260620 14:53:33.682173 15338 ts_tablet_manager.cc:1939] T e9481e4fac514e92b63d3521266d46f7 P 378c857c8c2741ef92b37e93b197fc91: Deleting consensus metadata
I20260620 14:53:33.682952 14786 catalog_manager.cc:4841] TS 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913): tablet e9481e4fac514e92b63d3521266d46f7 (table test-table [id=c5bb95a233004684a9a2cc69262c5212]) successfully deleted
I20260620 14:53:33.682987 15337 ts_tablet_manager.cc:1905] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:33.685696 15337 ts_tablet_manager.cc:1918] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:33.685779 15337 log.cc:1196] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/e9481e4fac514e92b63d3521266d46f7
I20260620 14:53:33.686022 15337 ts_tablet_manager.cc:1939] T e9481e4fac514e92b63d3521266d46f7 P f6946442599346978805099426058f15: Deleting consensus metadata
I20260620 14:53:33.686641 15336 ts_tablet_manager.cc:1905] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:33.686748 14787 catalog_manager.cc:4841] TS f6946442599346978805099426058f15 (127.7.24.194:46159): tablet e9481e4fac514e92b63d3521266d46f7 (table test-table [id=c5bb95a233004684a9a2cc69262c5212]) successfully deleted
I20260620 14:53:33.689985 15336 ts_tablet_manager.cc:1918] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:33.690075 15336 log.cc:1196] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/e9481e4fac514e92b63d3521266d46f7
I20260620 14:53:33.690357 15336 ts_tablet_manager.cc:1939] T e9481e4fac514e92b63d3521266d46f7 P f1ec9b72626a41299cf689b060de481b: Deleting consensus metadata
I20260620 14:53:33.691169 14788 catalog_manager.cc:4841] TS f1ec9b72626a41299cf689b060de481b (127.7.24.195:34005): tablet e9481e4fac514e92b63d3521266d46f7 (table test-table [id=c5bb95a233004684a9a2cc69262c5212]) successfully deleted
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[14757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
W20260620 14:53:33.712888 14851 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:33.705950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.7.24.254:38593)
0620 14:53:33.706109 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.706116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.706351 (+ 235us) server_negotiation.cc:407] Connection header received
0620 14:53:33.706607 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.706611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.706666 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.706758 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:33.707602 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.708132 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.709061 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.709287 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.709398 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.712734 (+ 3336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46656: BlockingRecv error: recv got EOF from 127.0.0.1:46656 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[14757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM
I20260620 14:53:33.723659 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.714485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46668 (local address 127.7.24.254:38593)
0620 14:53:33.714610 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.714614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.714848 (+ 234us) server_negotiation.cc:407] Connection header received
0620 14:53:33.714970 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.714973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.715012 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.715071 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:33.715850 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.716337 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.717394 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.717590 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.717662 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.720243 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:33.720247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:33.720250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:33.720276 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:33.721923 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:33.722340 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:33.722344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:33.722346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:33.722439 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:33.722748 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:33.722751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:33.722753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:33.722981 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:33.723113 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.723380 (+ 267us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.723498 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 14:53:33.726313 14801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:33.726819 14801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:33.731639 14939 tablet_service.cc:1467] Processing CreateTablet for tablet 57f91da7bd5746e3ad80f29717336ef3 (DEFAULT_TABLE table=test-table [id=76b85b0138ad4eec9c76de7661b954c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.731959 14939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f91da7bd5746e3ad80f29717336ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.733784 15317 tablet_bootstrap.cc:492] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91: Bootstrap starting.
I20260620 14:53:33.734150 15079 tablet_service.cc:1467] Processing CreateTablet for tablet 57f91da7bd5746e3ad80f29717336ef3 (DEFAULT_TABLE table=test-table [id=76b85b0138ad4eec9c76de7661b954c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.734411 15079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f91da7bd5746e3ad80f29717336ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.734666 15317 tablet_bootstrap.cc:654] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.735942 15317 tablet_bootstrap.cc:492] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91: No bootstrap required, opened a new log
I20260620 14:53:33.736035 15317 ts_tablet_manager.cc:1397] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:33.736904 15317 raft_consensus.cc:348] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.737020 15317 raft_consensus.cc:374] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.737059 15317 raft_consensus.cc:729] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378c857c8c2741ef92b37e93b197fc91, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.737172 15313 tablet_bootstrap.cc:492] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15: Bootstrap starting.
I20260620 14:53:33.737355 15317 consensus_queue.cc:260] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.737469 15321 raft_consensus.cc:482] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:33.737535 15321 raft_consensus.cc:504] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.738020 15313 tablet_bootstrap.cc:654] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.738762 15321 leader_election.cc:288] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6946442599346978805099426058f15 (127.7.24.194:46159), f1ec9b72626a41299cf689b060de481b (127.7.24.195:34005)
I20260620 14:53:33.738787 15219 tablet_service.cc:1467] Processing CreateTablet for tablet 57f91da7bd5746e3ad80f29717336ef3 (DEFAULT_TABLE table=test-table [id=76b85b0138ad4eec9c76de7661b954c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.738780 15317 ts_tablet_manager.cc:1428] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:33.739049 15219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f91da7bd5746e3ad80f29717336ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.741674 15314 tablet_bootstrap.cc:492] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b: Bootstrap starting.
I20260620 14:53:33.743832 15314 tablet_bootstrap.cc:654] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.744987 15313 tablet_bootstrap.cc:492] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15: No bootstrap required, opened a new log
I20260620 14:53:33.745074 15313 ts_tablet_manager.cc:1397] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260620 14:53:33.745342 15382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.739895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:46159 (local address 127.7.24.193:40455)
0620 14:53:33.740314 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.740332 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.740433 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.740888 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.740891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.740907 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:33.741075 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.741083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.742343 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.742349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.743658 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.743668 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.744696 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.744701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.744798 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.744823 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.745102 (+ 279us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.745186 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":65,"threads_started":1}
I20260620 14:53:33.745558 15313 raft_consensus.cc:348] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.745745 15313 raft_consensus.cc:374] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.745787 15313 raft_consensus.cc:729] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6946442599346978805099426058f15, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.745926 15313 consensus_queue.cc:260] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.746187 15313 ts_tablet_manager.cc:1428] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 14:53:33.746758 15381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.739706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:40455 (local address 127.7.24.194:46159)
0620 14:53:33.740174 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.740178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.740523 (+ 345us) server_negotiation.cc:407] Connection header received
0620 14:53:33.740579 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.740584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.740637 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.740716 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:33.741243 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.742180 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.743809 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.744430 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.744567 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.744633 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.744959 (+ 326us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.745043 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":82,"threads_started":1}
I20260620 14:53:33.747040 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f91da7bd5746e3ad80f29717336ef3" candidate_uuid: "378c857c8c2741ef92b37e93b197fc91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6946442599346978805099426058f15" is_pre_election: true
I20260620 14:53:33.747165 15099 raft_consensus.cc:2454] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378c857c8c2741ef92b37e93b197fc91 in term 0.
I20260620 14:53:33.747560 14891 leader_election.cc:302] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378c857c8c2741ef92b37e93b197fc91, f6946442599346978805099426058f15; no voters:
I20260620 14:53:33.748533 15314 tablet_bootstrap.cc:492] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b: No bootstrap required, opened a new log
I20260620 14:53:33.748629 15314 ts_tablet_manager.cc:1397] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260620 14:53:33.749078 15314 raft_consensus.cc:348] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.749083 15380 raft_consensus.cc:2781] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:33.749195 15380 raft_consensus.cc:482] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:33.749197 15314 raft_consensus.cc:374] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.749234 15380 raft_consensus.cc:3037] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.749236 15314 raft_consensus.cc:729] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ec9b72626a41299cf689b060de481b, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.749353 15314 consensus_queue.cc:260] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.749615 15314 ts_tablet_manager.cc:1428] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 14:53:33.750360 15380 raft_consensus.cc:504] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.750818 15380 leader_election.cc:288] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [CANDIDATE]: Term 1 election: Requested vote from peers f6946442599346978805099426058f15 (127.7.24.194:46159), f1ec9b72626a41299cf689b060de481b (127.7.24.195:34005)
I20260620 14:53:33.751073 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f91da7bd5746e3ad80f29717336ef3" candidate_uuid: "378c857c8c2741ef92b37e93b197fc91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6946442599346978805099426058f15"
I20260620 14:53:33.751176 15099 raft_consensus.cc:3037] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.751431 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.743066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:46079 (local address 127.7.24.195:34005)
0620 14:53:33.743163 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.743167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.743180 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:53:33.743220 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.743223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.743264 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.743339 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:33.744345 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.745109 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.747436 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.748063 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.748163 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.748212 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.751249 (+ 3037us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.751325 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 14:53:33.751754 15383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.742780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:34005 (local address 127.7.24.193:46079)
0620 14:53:33.743023 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.743038 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:33.743120 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:33.743358 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:33.743360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:33.743375 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:33.744218 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.744225 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.745371 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.745377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.747302 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:33.747314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.750980 (+ 3666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.750986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:33.751086 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.751110 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 14:53:33.751545 (+ 435us) client_negotiation.cc:240] Negotiation successful
0620 14:53:33.751622 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260620 14:53:33.752226 15099 raft_consensus.cc:2454] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378c857c8c2741ef92b37e93b197fc91 in term 1.
I20260620 14:53:33.752251 15239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f91da7bd5746e3ad80f29717336ef3" candidate_uuid: "378c857c8c2741ef92b37e93b197fc91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ec9b72626a41299cf689b060de481b" is_pre_election: true
I20260620 14:53:33.752271 15238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f91da7bd5746e3ad80f29717336ef3" candidate_uuid: "378c857c8c2741ef92b37e93b197fc91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ec9b72626a41299cf689b060de481b"
I20260620 14:53:33.752480 15239 raft_consensus.cc:2454] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378c857c8c2741ef92b37e93b197fc91 in term 0.
I20260620 14:53:33.752508 14891 leader_election.cc:302] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378c857c8c2741ef92b37e93b197fc91, f6946442599346978805099426058f15; no voters:
I20260620 14:53:33.752688 15380 raft_consensus.cc:2781] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:33.752772 15380 raft_consensus.cc:686] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 1 LEADER]: Becoming Leader. State: Replica: 378c857c8c2741ef92b37e93b197fc91, State: Running, Role: LEADER
I20260620 14:53:33.752980 15380 consensus_queue.cc:237] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.755234 14798 catalog_manager.cc:5495] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193). New cstate: current_term: 1 leader_uuid: "378c857c8c2741ef92b37e93b197fc91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } health_report { overall_health: UNKNOWN } } }
I20260620 14:53:33.768824 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.765234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.7.24.193:41913)
0620 14:53:33.765495 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.765501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.765513 (+ 12us) server_negotiation.cc:407] Connection header received
0620 14:53:33.765589 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.765592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.765643 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.765747 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:33.765939 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.766452 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.767354 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.767582 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.767676 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.767905 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:33.768352 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:33.768439 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.768533 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.768579 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":118,"threads_started":1}
I20260620 14:53:33.771448 15099 raft_consensus.cc:1261] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Refusing update from remote peer 378c857c8c2741ef92b37e93b197fc91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:33.771488 15238 raft_consensus.cc:3037] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.771920 15321 consensus_queue.cc:1035] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:33.772913 15238 raft_consensus.cc:1261] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Refusing update from remote peer 378c857c8c2741ef92b37e93b197fc91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:33.773361 15321 consensus_queue.cc:1035] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:33.783159 14798 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:33.783334 14798 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:33.785406 14798 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4: Sending DeleteTablet for 3 replicas of tablet 57f91da7bd5746e3ad80f29717336ef3
I20260620 14:53:33.786000 14939 tablet_service.cc:1514] Processing DeleteTablet for tablet 57f91da7bd5746e3ad80f29717336ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:57442
I20260620 14:53:33.786149 15079 tablet_service.cc:1514] Processing DeleteTablet for tablet 57f91da7bd5746e3ad80f29717336ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54594
I20260620 14:53:33.786217 15338 tablet_replica.cc:330] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91: stopping tablet replica
I20260620 14:53:33.786273 15219 tablet_service.cc:1514] Processing DeleteTablet for tablet 57f91da7bd5746e3ad80f29717336ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54540
I20260620 14:53:33.786327 15338 raft_consensus.cc:2229] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:33.786499 15336 tablet_replica.cc:330] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b: stopping tablet replica
I20260620 14:53:33.786626 15336 raft_consensus.cc:2229] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:33.786645 15338 raft_consensus.cc:2258] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:33.786755 15336 raft_consensus.cc:2258] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:33.786816 15337 tablet_replica.cc:330] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15: stopping tablet replica
I20260620 14:53:33.786923 15337 raft_consensus.cc:2229] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:33.787091 15337 raft_consensus.cc:2258] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:33.787336 14798 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 14:53:33.787645 15336 ts_tablet_manager.cc:1905] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 14:53:33.787760 14798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:33.787786 15338 ts_tablet_manager.cc:1905] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:33.790146 15336 ts_tablet_manager.cc:1918] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:33.790216 15336 log.cc:1196] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/57f91da7bd5746e3ad80f29717336ef3
I20260620 14:53:33.790227 15338 ts_tablet_manager.cc:1918] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:33.790299 15338 log.cc:1196] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-0/wal/wals/57f91da7bd5746e3ad80f29717336ef3
I20260620 14:53:33.790506 15336 ts_tablet_manager.cc:1939] T 57f91da7bd5746e3ad80f29717336ef3 P f1ec9b72626a41299cf689b060de481b: Deleting consensus metadata
I20260620 14:53:33.790596 15338 ts_tablet_manager.cc:1939] T 57f91da7bd5746e3ad80f29717336ef3 P 378c857c8c2741ef92b37e93b197fc91: Deleting consensus metadata
I20260620 14:53:33.791184 14788 catalog_manager.cc:4841] TS f1ec9b72626a41299cf689b060de481b (127.7.24.195:34005): tablet 57f91da7bd5746e3ad80f29717336ef3 (table test-table [id=76b85b0138ad4eec9c76de7661b954c3]) successfully deleted
I20260620 14:53:33.791234 14786 catalog_manager.cc:4841] TS 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913): tablet 57f91da7bd5746e3ad80f29717336ef3 (table test-table [id=76b85b0138ad4eec9c76de7661b954c3]) successfully deleted
I20260620 14:53:33.792646 14939 tablet_service.cc:1467] Processing CreateTablet for tablet 06aa60a238f34d238378f82265fd8825 (DEFAULT_TABLE table=test-table [id=62afb31910bc4ba9ad0809f704cbd432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.792647 15079 tablet_service.cc:1467] Processing CreateTablet for tablet 06aa60a238f34d238378f82265fd8825 (DEFAULT_TABLE table=test-table [id=62afb31910bc4ba9ad0809f704cbd432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.792692 15219 tablet_service.cc:1467] Processing CreateTablet for tablet 06aa60a238f34d238378f82265fd8825 (DEFAULT_TABLE table=test-table [id=62afb31910bc4ba9ad0809f704cbd432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:33.792892 15079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06aa60a238f34d238378f82265fd8825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.792892 14939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06aa60a238f34d238378f82265fd8825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.792928 15219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06aa60a238f34d238378f82265fd8825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:33.795086 15314 tablet_bootstrap.cc:492] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b: Bootstrap starting.
I20260620 14:53:33.795120 15317 tablet_bootstrap.cc:492] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91: Bootstrap starting.
I20260620 14:53:33.795943 15314 tablet_bootstrap.cc:654] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.796077 15317 tablet_bootstrap.cc:654] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.796461 15313 tablet_bootstrap.cc:492] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15: Bootstrap starting.
I20260620 14:53:33.797343 15337 ts_tablet_manager.cc:1905] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:33.797387 15313 tablet_bootstrap.cc:654] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:33.798846 15314 tablet_bootstrap.cc:492] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b: No bootstrap required, opened a new log
I20260620 14:53:33.798940 15314 ts_tablet_manager.cc:1397] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260620 14:53:33.799374 15314 raft_consensus.cc:348] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.799496 15314 raft_consensus.cc:374] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.799527 15314 raft_consensus.cc:729] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ec9b72626a41299cf689b060de481b, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.799615 15337 ts_tablet_manager.cc:1918] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:33.799692 15337 log.cc:1196] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/57f91da7bd5746e3ad80f29717336ef3
I20260620 14:53:33.799649 15314 consensus_queue.cc:260] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.799893 15314 ts_tablet_manager.cc:1428] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:33.799957 15337 ts_tablet_manager.cc:1939] T 57f91da7bd5746e3ad80f29717336ef3 P f6946442599346978805099426058f15: Deleting consensus metadata
I20260620 14:53:33.800676 14787 catalog_manager.cc:4841] TS f6946442599346978805099426058f15 (127.7.24.194:46159): tablet 57f91da7bd5746e3ad80f29717336ef3 (table test-table [id=76b85b0138ad4eec9c76de7661b954c3]) successfully deleted
I20260620 14:53:33.800856 15313 tablet_bootstrap.cc:492] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15: No bootstrap required, opened a new log
I20260620 14:53:33.800942 15313 ts_tablet_manager.cc:1397] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260620 14:53:33.801345 15313 raft_consensus.cc:348] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.801451 15313 raft_consensus.cc:374] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.801491 15313 raft_consensus.cc:729] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6946442599346978805099426058f15, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.801599 15313 consensus_queue.cc:260] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.801870 15313 ts_tablet_manager.cc:1428] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:33.803365 15317 tablet_bootstrap.cc:492] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91: No bootstrap required, opened a new log
I20260620 14:53:33.803464 15317 ts_tablet_manager.cc:1397] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260620 14:53:33.803960 15317 raft_consensus.cc:348] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.804112 15317 raft_consensus.cc:374] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:33.804147 15317 raft_consensus.cc:729] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378c857c8c2741ef92b37e93b197fc91, State: Initialized, Role: FOLLOWER
I20260620 14:53:33.804286 15317 consensus_queue.cc:260] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.804574 15317 ts_tablet_manager.cc:1428] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 14:53:33.827775 15320 raft_consensus.cc:482] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:33.827899 15320 raft_consensus.cc:504] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.828382 15320 leader_election.cc:288] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913), f6946442599346978805099426058f15 (127.7.24.194:46159)
I20260620 14:53:33.828758 14958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06aa60a238f34d238378f82265fd8825" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378c857c8c2741ef92b37e93b197fc91" is_pre_election: true
I20260620 14:53:33.828907 14958 raft_consensus.cc:2454] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 0.
I20260620 14:53:33.828884 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06aa60a238f34d238378f82265fd8825" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6946442599346978805099426058f15" is_pre_election: true
I20260620 14:53:33.829037 15099 raft_consensus.cc:2454] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 0.
I20260620 14:53:33.829211 15170 leader_election.cc:302] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378c857c8c2741ef92b37e93b197fc91, f1ec9b72626a41299cf689b060de481b; no voters:
I20260620 14:53:33.829397 15320 raft_consensus.cc:2781] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:33.829453 15320 raft_consensus.cc:482] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:33.829499 15320 raft_consensus.cc:3037] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.830714 15320 raft_consensus.cc:504] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.831074 15320 leader_election.cc:288] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 election: Requested vote from peers 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913), f6946442599346978805099426058f15 (127.7.24.194:46159)
I20260620 14:53:33.831399 14958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06aa60a238f34d238378f82265fd8825" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378c857c8c2741ef92b37e93b197fc91"
I20260620 14:53:33.831386 15099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06aa60a238f34d238378f82265fd8825" candidate_uuid: "f1ec9b72626a41299cf689b060de481b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6946442599346978805099426058f15"
I20260620 14:53:33.831514 14958 raft_consensus.cc:3037] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.831540 15099 raft_consensus.cc:3037] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:33.832547 15099 raft_consensus.cc:2454] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 1.
I20260620 14:53:33.832739 14958 raft_consensus.cc:2454] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ec9b72626a41299cf689b060de481b in term 1.
I20260620 14:53:33.832923 15171 leader_election.cc:302] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f1ec9b72626a41299cf689b060de481b, f6946442599346978805099426058f15; no voters:
I20260620 14:53:33.833115 15320 raft_consensus.cc:2781] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:33.833209 15320 raft_consensus.cc:686] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 1 LEADER]: Becoming Leader. State: Replica: f1ec9b72626a41299cf689b060de481b, State: Running, Role: LEADER
I20260620 14:53:33.833349 15320 consensus_queue.cc:237] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } }
I20260620 14:53:33.835192 14798 catalog_manager.cc:5495] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b reported cstate change: term changed from 0 to 1, leader changed from <none> to f1ec9b72626a41299cf689b060de481b (127.7.24.195). New cstate: current_term: 1 leader_uuid: "f1ec9b72626a41299cf689b060de481b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1ec9b72626a41299cf689b060de481b" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34005 } health_report { overall_health: HEALTHY } } }
I20260620 14:53:33.859251 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.853625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54576 (local address 127.7.24.195:34005)
0620 14:53:33.853886 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:33.853894 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:33.853910 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:53:33.853959 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:33.853964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:33.854027 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:33.854102 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:33.854849 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.856009 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.857933 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:33.858284 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:33.858381 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:33.858693 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:33.858781 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:33.858857 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:33.859045 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 14:53:33.859097 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260620 14:53:33.866181 15099 raft_consensus.cc:1261] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Refusing update from remote peer f1ec9b72626a41299cf689b060de481b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:33.866572 14958 raft_consensus.cc:1261] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Refusing update from remote peer f1ec9b72626a41299cf689b060de481b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:33.866590 15395 consensus_queue.cc:1035] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6946442599346978805099426058f15" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:33.866951 15395 consensus_queue.cc:1035] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [LEADER]: Connected to new peer: Peer: permanent_uuid: "378c857c8c2741ef92b37e93b197fc91" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 41913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:33.904822 14801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:33.904990 14801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:33.906997 14801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bfcdf28c056408a9ff6c748d7d7f3b4: Sending DeleteTablet for 3 replicas of tablet 06aa60a238f34d238378f82265fd8825
I20260620 14:53:33.907492 14939 tablet_service.cc:1514] Processing DeleteTablet for tablet 06aa60a238f34d238378f82265fd8825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:57442
I20260620 14:53:33.907861 15338 tablet_replica.cc:330] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91: stopping tablet replica
I20260620 14:53:33.908000 15338 raft_consensus.cc:2229] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:33.908124 15338 raft_consensus.cc:2258] T 06aa60a238f34d238378f82265fd8825 P 378c857c8c2741ef92b37e93b197fc91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:33.908243 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 14874
I20260620 14:53:33.908313 15218 tablet_service.cc:1514] Processing DeleteTablet for tablet 06aa60a238f34d238378f82265fd8825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54540
I20260620 14:53:33.908489 15079 tablet_service.cc:1514] Processing DeleteTablet for tablet 06aa60a238f34d238378f82265fd8825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:33 UTC) from {username='oryx', principal='oryx/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54594
I20260620 14:53:33.908871 15336 tablet_replica.cc:330] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b: stopping tablet replica
I20260620 14:53:33.908993 15336 raft_consensus.cc:2229] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:33.909209 15336 raft_consensus.cc:2258] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:33.909461 15337 tablet_replica.cc:330] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15: stopping tablet replica
I20260620 14:53:33.909598 15337 raft_consensus.cc:2229] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:33.909727 15337 raft_consensus.cc:2258] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:33.910900 15337 ts_tablet_manager.cc:1905] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:33.910949 15336 ts_tablet_manager.cc:1905] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:33.912810 15337 ts_tablet_manager.cc:1918] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:33.912904 15337 log.cc:1196] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/06aa60a238f34d238378f82265fd8825
I20260620 14:53:33.913151 15337 ts_tablet_manager.cc:1939] T 06aa60a238f34d238378f82265fd8825 P f6946442599346978805099426058f15: Deleting consensus metadata
I20260620 14:53:33.913794 14787 catalog_manager.cc:4841] TS f6946442599346978805099426058f15 (127.7.24.194:46159): tablet 06aa60a238f34d238378f82265fd8825 (table test-table [id=62afb31910bc4ba9ad0809f704cbd432]) successfully deleted
I20260620 14:53:33.914058 15336 ts_tablet_manager.cc:1918] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 14:53:33.914152 15336 log.cc:1196] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/06aa60a238f34d238378f82265fd8825
I20260620 14:53:33.914412 15336 ts_tablet_manager.cc:1939] T 06aa60a238f34d238378f82265fd8825 P f1ec9b72626a41299cf689b060de481b: Deleting consensus metadata
I20260620 14:53:33.915185 14788 catalog_manager.cc:4841] TS f1ec9b72626a41299cf689b060de481b (127.7.24.195:34005): tablet 06aa60a238f34d238378f82265fd8825 (table test-table [id=62afb31910bc4ba9ad0809f704cbd432]) successfully deleted
W20260620 14:53:33.917474 14786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:41913 (error 108)
I20260620 14:53:33.917625 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 15014
I20260620 14:53:33.919761 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:33.919406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:41913 (local address 127.0.0.1:57462)
0620 14:53:33.919623 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:33.919683 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:41913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
W20260620 14:53:33.919886 14786 catalog_manager.cc:4584] TS 378c857c8c2741ef92b37e93b197fc91 (127.7.24.193:41913): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06aa60a238f34d238378f82265fd8825: Network error: Client connection negotiation failed: client connection to 127.7.24.193:41913: connect: Connection refused (error 111)
I20260620 14:53:33.923779 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 15154
I20260620 14:53:33.930573 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 14770
2026-06-20T14:53:33Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2930 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[15402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[15402](info): set up 2 sockets
Jun 20 14:53:33 dist-test-slave-sms9 krb5kdc[15402](info): commencing operation
krb5kdc: starting...
W20260620 14:53:34.971277 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:35 dist-test-slave-sms9 krb5kdc[15402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:35.097478 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:34621
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:34621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:35.208335 15415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:35.208633 15415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:35.208675 15415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:35.212812 15415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:35.212889 15415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:35.212909 15415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:35.212927 15415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:35.212942 15415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:35.218436 15415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:34621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:34621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:35.219663 15415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:35.220721 15415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:35.229758 15420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:35.229882 15415 server_base.cc:1034] running on GCE node
W20260620 14:53:35.229758 15421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:35.229758 15423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:35.230804 15415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:35.231926 15415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:35.233129 15415 hybrid_clock.cc:648] HybridClock initialized: now 1781967215233109 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-20T14:53:35Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1314 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:35 dist-test-slave-sms9 krb5kdc[15428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:35 dist-test-slave-sms9 krb5kdc[15428](info): set up 2 sockets
Jun 20 14:53:35 dist-test-slave-sms9 krb5kdc[15428](info): commencing operation
krb5kdc: starting...
W20260620 14:53:36.323231 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:36 dist-test-slave-sms9 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:36.449918 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:42011
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:42011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:36.559935 15441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:36.560249 15441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:36.560323 15441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:36.564414 15441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:36.564500 15441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:36.564538 15441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:36.564595 15441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:36.564635 15441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:36.569947 15441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:42011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:42011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:36.571100 15441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:36.572118 15441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:36.580494 15446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:36.580502 15449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:36.580494 15447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:36.580547 15441 server_base.cc:1034] running on GCE node
I20260620 14:53:36.581391 15441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:36.582527 15441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:36.583755 15441 hybrid_clock.cc:648] HybridClock initialized: now 1781967216583742 us; error 36 us; skew 500 ppm
Jun 20 14:53:36 dist-test-slave-sms9 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:36.592628 15441 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:36.593689 15441 webserver.cc:460] Webserver started at http://127.7.24.254:38951/ using document root <none> and password file <none>
I20260620 14:53:36.594308 15441 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:36.594370 15441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:36.594710 15441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:36.596793 15441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "70262117fc5f4bb5a7bc0f949e971a21"
format_stamp: "Formatted at 2026-06-20 14:53:36 on dist-test-slave-sms9"
server_key: "b20a9cc04c204906ca238ef0c7a7ebbd"
server_key_iv: "eb3633a0a5e07000edf5ae36d2f04a60"
server_key_version: "encryptionkey@0"
I20260620 14:53:36.597302 15441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "70262117fc5f4bb5a7bc0f949e971a21"
format_stamp: "Formatted at 2026-06-20 14:53:36 on dist-test-slave-sms9"
server_key: "b20a9cc04c204906ca238ef0c7a7ebbd"
server_key_iv: "eb3633a0a5e07000edf5ae36d2f04a60"
server_key_version: "encryptionkey@0"
I20260620 14:53:36.600975 15441 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 14:53:36.603422 15456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:36.604738 15441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260620 14:53:36.604882 15441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "70262117fc5f4bb5a7bc0f949e971a21"
format_stamp: "Formatted at 2026-06-20 14:53:36 on dist-test-slave-sms9"
server_key: "b20a9cc04c204906ca238ef0c7a7ebbd"
server_key_iv: "eb3633a0a5e07000edf5ae36d2f04a60"
server_key_version: "encryptionkey@0"
I20260620 14:53:36.604993 15441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:36.627364 15441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:36.630388 15441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:36.630645 15441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:36.638032 15441 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:42011
I20260620 14:53:36.638048 15508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:42011 every 8 connection(s)
I20260620 14:53:36.639070 15441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:36.641822 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:36.646684 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 15441
I20260620 14:53:36.646818 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:36.647127 7267 external_mini_cluster.cc:1428] Setting key 9820b6ea660a632ce009a4daed8dc197
I20260620 14:53:36.647764 15509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21: Bootstrap starting.
I20260620 14:53:36.650199 15509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:36.651024 15509 log.cc:824] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:36.652911 15509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21: No bootstrap required, opened a new log
Jun 20 14:53:36 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:36.656716 15509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } }
I20260620 14:53:36.657024 15509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:36.657115 15509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70262117fc5f4bb5a7bc0f949e971a21, State: Initialized, Role: FOLLOWER
I20260620 14:53:36.657648 15509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } }
I20260620 14:53:36.657821 15509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:36.657917 15509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:36.658015 15509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:36.659008 15509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } }
I20260620 14:53:36.659366 15509 leader_election.cc:302] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70262117fc5f4bb5a7bc0f949e971a21; no voters:
I20260620 14:53:36.659679 15509 leader_election.cc:288] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:36.659786 15514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:36.660010 15514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 1 LEADER]: Becoming Leader. State: Replica: 70262117fc5f4bb5a7bc0f949e971a21, State: Running, Role: LEADER
I20260620 14:53:36.660377 15514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } }
I20260620 14:53:36.661109 15509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:36.661592 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:36.648678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35100 (local address 127.7.24.254:42011)
0620 14:53:36.649094 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:36.649119 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:36.649154 (+ 35us) server_negotiation.cc:407] Connection header received
0620 14:53:36.649785 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:36.649802 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:36.650131 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:36.650496 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:36.651354 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:36.652302 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:36.653227 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:36.653690 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:36.653859 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:36.656403 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:36.656409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:36.656422 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:36.656450 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:36.658367 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:36.658845 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:36.658851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:36.658864 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:36.658971 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:36.659272 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:36.659276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:36.659278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:36.659818 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:36.660066 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:36.660323 (+ 257us) server_negotiation.cc:299] Negotiation successful
0620 14:53:36.660648 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":132,"threads_started":1}
I20260620 14:53:36.661653 15515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70262117fc5f4bb5a7bc0f949e971a21. Latest consensus state: current_term: 1 leader_uuid: "70262117fc5f4bb5a7bc0f949e971a21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } } }
I20260620 14:53:36.661774 15515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:36.661931 15516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70262117fc5f4bb5a7bc0f949e971a21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } } }
I20260620 14:53:36.662020 15516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:36.662708 15518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:36.665334 15518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:36.671541 15518 catalog_manager.cc:1347] Generated new cluster ID: 8ef0f578554d491894a452fcaf8b083b
I20260620 14:53:36.671655 15518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:36.693296 15518 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:36.694376 15518 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:36.700861 15518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21: Generated new TSK 0
I20260620 14:53:36.701596 15518 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:53:36.737264 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42011
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:36.849742 15535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:36.850005 15535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:36.850059 15535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:36.854105 15535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:36.854162 15535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:36.854221 15535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:36.859385 15535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:36.860370 15535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:36.861366 15535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:36.872010 15543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:36.872035 15541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:36.871996 15540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:36.872474 15535 server_base.cc:1034] running on GCE node
I20260620 14:53:36.872861 15535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:36.873588 15535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:36.874758 15535 hybrid_clock.cc:648] HybridClock initialized: now 1781967216874740 us; error 31 us; skew 500 ppm
Jun 20 14:53:36 dist-test-slave-sms9 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:36.877637 15535 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:53:36.878686 15535 webserver.cc:460] Webserver started at http://127.7.24.193:38809/ using document root <none> and password file <none>
I20260620 14:53:36.879245 15535 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:36.879304 15535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:36.879532 15535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:36.881573 15535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "7f150a8c809d4b2b817e8e8ff7dbf1fb"
format_stamp: "Formatted at 2026-06-20 14:53:36 on dist-test-slave-sms9"
server_key: "b8e0c5c5fa05427baf0a760b20412a74"
server_key_iv: "985842bdfed90d88719972e05dcba945"
server_key_version: "encryptionkey@0"
I20260620 14:53:36.882118 15535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "7f150a8c809d4b2b817e8e8ff7dbf1fb"
format_stamp: "Formatted at 2026-06-20 14:53:36 on dist-test-slave-sms9"
server_key: "b8e0c5c5fa05427baf0a760b20412a74"
server_key_iv: "985842bdfed90d88719972e05dcba945"
server_key_version: "encryptionkey@0"
I20260620 14:53:36.885877 15535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260620 14:53:36.888490 15550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:36.889854 15535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:36.889966 15535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "7f150a8c809d4b2b817e8e8ff7dbf1fb"
format_stamp: "Formatted at 2026-06-20 14:53:36 on dist-test-slave-sms9"
server_key: "b8e0c5c5fa05427baf0a760b20412a74"
server_key_iv: "985842bdfed90d88719972e05dcba945"
server_key_version: "encryptionkey@0"
I20260620 14:53:36.890058 15535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:36.926766 15535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:36.930335 15535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:36.930579 15535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:36.931177 15535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:36.932014 15535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:36.932065 15535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:36.932106 15535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:36.932124 15535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:36.942059 15535 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:36419
I20260620 14:53:36.942085 15663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:36419 every 8 connection(s)
I20260620 14:53:36.943167 15535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:36.944303 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 15535
I20260620 14:53:36.944458 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:36.944764 7267 external_mini_cluster.cc:1428] Setting key 92caefefd02f685185205c210a6b005e
Jun 20 14:53:36 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
I20260620 14:53:36.958297 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:36.945452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:57959 (local address 127.7.24.254:42011)
0620 14:53:36.945645 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:36.945649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:36.946565 (+ 916us) server_negotiation.cc:407] Connection header received
0620 14:53:36.947918 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:36.947922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:36.947978 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:36.948091 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:36.949397 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:36.949929 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:36.951179 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:36.951410 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:36.951528 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:36.954526 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:36.954529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:36.954532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:36.954563 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:36.956078 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:36.956615 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:36.956623 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:36.956625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:36.956735 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:36.957120 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:36.957126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:36.957130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:36.957370 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:36.957476 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:36.957994 (+ 518us) server_negotiation.cc:299] Negotiation successful
0620 14:53:36.958129 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 14:53:36.959431 15666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:36.945878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:57959)
0620 14:53:36.946364 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:36.946411 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:36.947692 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:36.948319 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:36.948335 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:36.948752 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:36.949238 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:36.949251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:36.950065 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:36.950070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:36.951042 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:36.951052 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:36.951577 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:36.951581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:36.951755 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:36.952460 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:36.952477 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:36.954293 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:36.956201 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:36.956207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:36.956225 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:36.956491 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:36.956851 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:36.956854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:36.956856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:36.956998 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:36.957485 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:36.957495 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:36.957757 (+ 262us) client_negotiation.cc:769] Sending connection context
0620 14:53:36.957956 (+ 199us) client_negotiation.cc:240] Negotiation successful
0620 14:53:36.958298 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":133,"threads_started":1}
I20260620 14:53:36.960783 15664 heartbeater.cc:347] Connected to a master server at 127.7.24.254:42011
I20260620 14:53:36.961045 15664 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:36.961596 15664 heartbeater.cc:511] Master 127.7.24.254:42011 requested a full tablet report, sending...
I20260620 14:53:36.963501 15473 ts_manager.cc:194] Registered new tserver with Master: 7f150a8c809d4b2b817e8e8ff7dbf1fb (127.7.24.193:36419)
I20260620 14:53:36.965181 15473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:57959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:53:36.975741 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42011
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:37.088289 15669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:37.088562 15669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:37.088611 15669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:37.092744 15669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:37.092810 15669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:37.092871 15669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:37.098299 15669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:37.099380 15669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:37.100340 15669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:37.109941 15677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:37.109958 15675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:37.109951 15674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:37.110390 15669 server_base.cc:1034] running on GCE node
I20260620 14:53:37.110806 15669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:37.111456 15669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:37.112658 15669 hybrid_clock.cc:648] HybridClock initialized: now 1781967217112630 us; error 47 us; skew 500 ppm
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967217, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:37.115465 15669 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:53:37.116429 15669 webserver.cc:460] Webserver started at http://127.7.24.194:40187/ using document root <none> and password file <none>
I20260620 14:53:37.117017 15669 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:37.117112 15669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:37.117342 15669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:37.119491 15669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "6b0391796c544c3e91c3a8d0c1f65ead"
format_stamp: "Formatted at 2026-06-20 14:53:37 on dist-test-slave-sms9"
server_key: "988285621c63db493bb2774d1c62878b"
server_key_iv: "103217c217e9d8638bf9d0aef077b9c0"
server_key_version: "encryptionkey@0"
I20260620 14:53:37.120080 15669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "6b0391796c544c3e91c3a8d0c1f65ead"
format_stamp: "Formatted at 2026-06-20 14:53:37 on dist-test-slave-sms9"
server_key: "988285621c63db493bb2774d1c62878b"
server_key_iv: "103217c217e9d8638bf9d0aef077b9c0"
server_key_version: "encryptionkey@0"
I20260620 14:53:37.123704 15669 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:53:37.126088 15684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:37.127298 15669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:37.127439 15669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "6b0391796c544c3e91c3a8d0c1f65ead"
format_stamp: "Formatted at 2026-06-20 14:53:37 on dist-test-slave-sms9"
server_key: "988285621c63db493bb2774d1c62878b"
server_key_iv: "103217c217e9d8638bf9d0aef077b9c0"
server_key_version: "encryptionkey@0"
I20260620 14:53:37.127548 15669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:37.146207 15669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:37.149389 15669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:37.149605 15669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:37.150204 15669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:37.151100 15669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:37.151175 15669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:37.151245 15669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:37.151297 15669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:37.160059 15669 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:35721
I20260620 14:53:37.160069 15797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:35721 every 8 connection(s)
I20260620 14:53:37.161000 15669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:37.162554 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 15669
I20260620 14:53:37.162673 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:37.163002 7267 external_mini_cluster.cc:1428] Setting key b2a8af483649f16311985d673648ada1
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967217, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
I20260620 14:53:37.177161 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:37.162959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:38189 (local address 127.7.24.254:42011)
0620 14:53:37.163109 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:37.163113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:37.164091 (+ 978us) server_negotiation.cc:407] Connection header received
0620 14:53:37.165424 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:37.165427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:37.165488 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:37.165569 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:37.167353 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.167985 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.169847 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.170060 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.170143 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:37.173159 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:37.173162 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:37.173164 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:37.173188 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:37.174769 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:37.175382 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:37.175388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:37.175392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:37.175477 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:37.175853 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:37.175858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:37.175859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:37.176080 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:37.176195 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:37.176888 (+ 693us) server_negotiation.cc:299] Negotiation successful
0620 14:53:37.177003 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 14:53:37.178349 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:37.163469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:38189)
0620 14:53:37.163933 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:37.163975 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:37.165215 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:37.165754 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:37.165761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:37.166404 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:37.167161 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:37.167185 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.168126 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.168133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:37.169717 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:37.169733 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.170230 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.170237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:37.170509 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:37.171156 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:37.171184 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:37.172962 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:37.174892 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:37.174899 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:37.174915 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:37.175276 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:37.175592 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:37.175596 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:37.175598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:37.175757 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:37.176203 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:37.176212 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:37.176578 (+ 366us) client_negotiation.cc:769] Sending connection context
0620 14:53:37.176810 (+ 232us) client_negotiation.cc:240] Negotiation successful
0620 14:53:37.177211 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260620 14:53:37.179749 15798 heartbeater.cc:347] Connected to a master server at 127.7.24.254:42011
I20260620 14:53:37.180018 15798 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:37.180560 15798 heartbeater.cc:511] Master 127.7.24.254:42011 requested a full tablet report, sending...
I20260620 14:53:37.181732 15473 ts_manager.cc:194] Registered new tserver with Master: 6b0391796c544c3e91c3a8d0c1f65ead (127.7.24.194:35721)
I20260620 14:53:37.182366 15473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:38189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:53:37.193670 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42011
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:37.305136 15803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:37.305455 15803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:37.305537 15803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:37.310051 15803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:37.310137 15803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:37.310225 15803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:37.315833 15803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:37.316908 15803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:37.317909 15803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:37.327661 15811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:37.327679 15808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:37.327668 15809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:37.328003 15803 server_base.cc:1034] running on GCE node
I20260620 14:53:37.328467 15803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:37.329073 15803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:37.330232 15803 hybrid_clock.cc:648] HybridClock initialized: now 1781967217330190 us; error 52 us; skew 500 ppm
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967217, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:37.333076 15803 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:53:37.334091 15803 webserver.cc:460] Webserver started at http://127.7.24.195:37241/ using document root <none> and password file <none>
I20260620 14:53:37.334717 15803 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:37.334805 15803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:37.335050 15803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:37.337051 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "cd4bde19722645bdb09531b5491285cb"
format_stamp: "Formatted at 2026-06-20 14:53:37 on dist-test-slave-sms9"
server_key: "a94beaa204796ae19e3def9a351aa58d"
server_key_iv: "82d208586e99ec04765323619978159c"
server_key_version: "encryptionkey@0"
I20260620 14:53:37.337613 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "cd4bde19722645bdb09531b5491285cb"
format_stamp: "Formatted at 2026-06-20 14:53:37 on dist-test-slave-sms9"
server_key: "a94beaa204796ae19e3def9a351aa58d"
server_key_iv: "82d208586e99ec04765323619978159c"
server_key_version: "encryptionkey@0"
I20260620 14:53:37.341288 15803 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 14:53:37.343813 15818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:37.345019 15803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:53:37.345191 15803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "cd4bde19722645bdb09531b5491285cb"
format_stamp: "Formatted at 2026-06-20 14:53:37 on dist-test-slave-sms9"
server_key: "a94beaa204796ae19e3def9a351aa58d"
server_key_iv: "82d208586e99ec04765323619978159c"
server_key_version: "encryptionkey@0"
I20260620 14:53:37.345307 15803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:37.374791 15803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:37.377807 15803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:37.378015 15803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:37.378683 15803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:37.379515 15803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:37.379590 15803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:37.379671 15803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:37.379722 15803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:37.388841 15803 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:37137
I20260620 14:53:37.388860 15931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:37137 every 8 connection(s)
I20260620 14:53:37.389740 15803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:37.390482 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 15803
I20260620 14:53:37.390643 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:37.390920 7267 external_mini_cluster.cc:1428] Setting key 8361c0882e5340cbb417c5b01f308fa7
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967217, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:37.402936 15512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:37.391874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:56973 (local address 127.7.24.254:42011)
0620 14:53:37.391999 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:37.392003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:37.392779 (+ 776us) server_negotiation.cc:407] Connection header received
0620 14:53:37.394188 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:37.394191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:37.394245 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:37.394346 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:37.395558 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.396092 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.397206 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.397436 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.397543 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:37.399960 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:37.399963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:37.399965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:37.399988 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:37.401198 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:37.401641 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:37.401643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:37.401644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:37.401712 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:37.401991 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:37.401993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:37.401994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:37.402167 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:37.402239 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:37.402724 (+ 485us) server_negotiation.cc:299] Negotiation successful
0620 14:53:37.402816 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:53:37.403930 15934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:37.392194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:56973)
0620 14:53:37.392647 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:37.392689 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:37.393950 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:37.394554 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:37.394561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:37.394961 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:37.395413 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:37.395425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.396219 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.396222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:37.397084 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:37.397093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.397576 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.397579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:37.397750 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:37.398267 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:37.398283 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:37.399814 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:37.401299 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:37.401304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:37.401315 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:37.401555 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:37.401790 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:37.401792 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:37.401793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:37.401915 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:37.402248 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:37.402253 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:37.402511 (+ 258us) client_negotiation.cc:769] Sending connection context
0620 14:53:37.402677 (+ 166us) client_negotiation.cc:240] Negotiation successful
0620 14:53:37.402921 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":137,"threads_started":1}
I20260620 14:53:37.405057 15932 heartbeater.cc:347] Connected to a master server at 127.7.24.254:42011
I20260620 14:53:37.405267 15932 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:37.405680 15932 heartbeater.cc:511] Master 127.7.24.254:42011 requested a full tablet report, sending...
I20260620 14:53:37.406653 15472 ts_manager.cc:194] Registered new tserver with Master: cd4bde19722645bdb09531b5491285cb (127.7.24.195:37137)
I20260620 14:53:37.407217 15472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:56973
I20260620 14:53:37.416663 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab.
Entry for principal oryx/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab.
WARNING: no policy specified for oryx/127.7.24.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab.
Entry for principal oryx/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab.
WARNING: no policy specified for oryx/127.7.24.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.24.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab.
Entry for principal oryx/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab.
I20260620 14:53:37.490595 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 15535
I20260620 14:53:37.498085 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 15669
I20260620 14:53:37.505488 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 15803
I20260620 14:53:37.510905 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 15441
I20260620 14:53:37.517093 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:42011
--webserver_interface=127.7.24.254
--webserver_port=38951
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:42011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:37.625098 15941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:37.625408 15941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:37.625483 15941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:37.629559 15941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:37.629657 15941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:37.629694 15941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:37.629750 15941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:37.629786 15941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:37.635171 15941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:42011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:42011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=38951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:37.636279 15941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:37.637286 15941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:37.645622 15949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:37.645640 15941 server_base.cc:1034] running on GCE node
W20260620 14:53:37.645726 15946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:37.645632 15947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:37.646364 15941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:37.647485 15941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:37.648686 15941 hybrid_clock.cc:648] HybridClock initialized: now 1781967217648657 us; error 39 us; skew 500 ppm
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967217, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:37.651544 15941 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:37.652325 15941 webserver.cc:460] Webserver started at http://127.7.24.254:38951/ using document root <none> and password file <none>
I20260620 14:53:37.652875 15941 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:37.652947 15941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:37.656695 15941 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260620 14:53:37.658785 15956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:37.660043 15941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 14:53:37.660174 15941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "70262117fc5f4bb5a7bc0f949e971a21"
format_stamp: "Formatted at 2026-06-20 14:53:36 on dist-test-slave-sms9"
server_key: "b20a9cc04c204906ca238ef0c7a7ebbd"
server_key_iv: "eb3633a0a5e07000edf5ae36d2f04a60"
server_key_version: "encryptionkey@0"
I20260620 14:53:37.660704 15941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:37.693913 15941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:37.697495 15941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:37.697670 15941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:37.705464 15941 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:42011
I20260620 14:53:37.705488 16008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:42011 every 8 connection(s)
I20260620 14:53:37.706703 15941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:37.711375 16009 sys_catalog.cc:263] Verifying existing consensus state
I20260620 14:53:37.713129 16009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21: Bootstrap starting.
I20260620 14:53:37.713596 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 15941
I20260620 14:53:37.714077 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:36419
--local_ip_for_outbound_sockets=127.7.24.193
--tserver_master_addrs=127.7.24.254:42011
--webserver_port=38809
--webserver_interface=127.7.24.193
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.24.193
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:53:37.736330 16009 log.cc:824] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:37.745117 16009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:53:37.745764 16009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21: Bootstrap complete.
I20260620 14:53:37.751665 16009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } }
I20260620 14:53:37.752213 16009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70262117fc5f4bb5a7bc0f949e971a21, State: Initialized, Role: FOLLOWER
I20260620 14:53:37.752774 16009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } }
I20260620 14:53:37.752918 16009 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:37.752983 16009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:37.753085 16009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 1 FOLLOWER]: Advancing to term 2
I20260620 14:53:37.755738 16009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } }
I20260620 14:53:37.756121 16009 leader_election.cc:302] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70262117fc5f4bb5a7bc0f949e971a21; no voters:
I20260620 14:53:37.756469 16009 leader_election.cc:288] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 14:53:37.756767 16013 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 14:53:37.757017 16013 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [term 2 LEADER]: Becoming Leader. State: Replica: 70262117fc5f4bb5a7bc0f949e971a21, State: Running, Role: LEADER
I20260620 14:53:37.757354 16013 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } }
I20260620 14:53:37.757756 16009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:37.759105 16014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "70262117fc5f4bb5a7bc0f949e971a21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } } }
I20260620 14:53:37.759262 16014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:37.759083 16015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70262117fc5f4bb5a7bc0f949e971a21. Latest consensus state: current_term: 2 leader_uuid: "70262117fc5f4bb5a7bc0f949e971a21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70262117fc5f4bb5a7bc0f949e971a21" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42011 } } }
I20260620 14:53:37.759390 16015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:37.759684 16023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:37.762912 16023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:37.763641 16023 catalog_manager.cc:1259] Loaded cluster ID: 8ef0f578554d491894a452fcaf8b083b
I20260620 14:53:37.763702 16023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:37.766374 16023 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:37.767153 16023 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 70262117fc5f4bb5a7bc0f949e971a21: Loaded TSK: 0
I20260620 14:53:37.768587 16023 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 14:53:37.844187 16011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:37.844493 16011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:37.844579 16011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:37.848836 16011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:37.848937 16011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:37.849027 16011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:37.854638 16011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.193.keytab
--principal=oryx/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:36419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=38809
--tserver_master_addrs=127.7.24.254:42011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:37.855723 16011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:37.856731 16011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:37.867825 16035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:37.867885 16036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:37.867832 16038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:37.868238 16011 server_base.cc:1034] running on GCE node
I20260620 14:53:37.868670 16011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:37.869454 16011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:37.870623 16011 hybrid_clock.cc:648] HybridClock initialized: now 1781967217870598 us; error 43 us; skew 500 ppm
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967217, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:37.873551 16011 init.cc:377] Logged in from keytab as oryx/127.7.24.193@KRBTEST.COM (short username oryx)
I20260620 14:53:37.874589 16011 webserver.cc:460] Webserver started at http://127.7.24.193:38809/ using document root <none> and password file <none>
I20260620 14:53:37.875205 16011 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:37.875316 16011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:37.879387 16011 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:53:37.881495 16045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:37.882848 16011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:37.882988 16011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "7f150a8c809d4b2b817e8e8ff7dbf1fb"
format_stamp: "Formatted at 2026-06-20 14:53:36 on dist-test-slave-sms9"
server_key: "b8e0c5c5fa05427baf0a760b20412a74"
server_key_iv: "985842bdfed90d88719972e05dcba945"
server_key_version: "encryptionkey@0"
I20260620 14:53:37.883534 16011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:37.903795 16011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:37.907161 16011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:37.907383 16011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:37.908006 16011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:37.908830 16011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:37.908876 16011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:37.908949 16011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:37.908979 16011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:37.917836 16011 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:36419
I20260620 14:53:37.917870 16158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:36419 every 8 connection(s)
I20260620 14:53:37.918852 16011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:37.921972 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 16011
I20260620 14:53:37.922564 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:35721
--local_ip_for_outbound_sockets=127.7.24.194
--tserver_master_addrs=127.7.24.254:42011
--webserver_port=40187
--webserver_interface=127.7.24.194
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.24.194
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
W20260620 14:53:37.933717 16161 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:37.920806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:52823 (local address 127.7.24.254:42011)
0620 14:53:37.921295 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:37.921317 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:37.921562 (+ 245us) server_negotiation.cc:407] Connection header received
0620 14:53:37.923666 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:37.923678 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:37.924053 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:37.924424 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:37.925459 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.927197 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.928319 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.929132 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.929344 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:37.933019 (+ 3675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:52823: BlockingRecv error: recv got EOF from 127.7.24.193:52823 (error 108)
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":123,"threads_started":1}
I20260620 14:53:37.933838 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:37.920930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:52823)
0620 14:53:37.921412 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:37.921454 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:37.923268 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:37.924488 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:37.924494 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:37.924843 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:37.925288 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:37.925301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.927337 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.927340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:37.928174 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:37.928183 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.929299 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.929302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:37.929433 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:37.930056 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:37.930072 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:37.932833 (+ 2761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":129,"threads_started":1}
W20260620 14:53:37.934283 16159 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:42011 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.24.254:42011: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:37.940269 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:37.934807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:39219)
0620 14:53:37.934928 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:37.934943 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:37.935020 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:37.935245 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:37.935248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:37.935416 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:37.935545 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:37.935551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.936292 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.936295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:37.937193 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:37.937204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.937618 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.937621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:37.937714 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:37.938172 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:37.938192 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:37.940088 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 14:53:37.940305 16161 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:37.934906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:39219 (local address 127.7.24.254:42011)
0620 14:53:37.935014 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:37.935017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:37.935029 (+ 12us) server_negotiation.cc:407] Connection header received
0620 14:53:37.935102 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:37.935104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:37.935141 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:37.935214 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:37.935668 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.936172 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.937311 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.937515 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.937592 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:37.940220 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:39219: BlockingRecv error: recv got EOF from 127.7.24.193:39219 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:37 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:37.945996 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:37.940905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:40111)
0620 14:53:37.941099 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:37.941120 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:37.941179 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:37.941398 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:37.941400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:37.941519 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:37.941586 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:37.941591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.942570 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.942572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:37.943084 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:37.943091 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.943639 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.943641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:37.943715 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:37.944188 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:37.944202 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:37.945907 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129}
W20260620 14:53:37.946167 16161 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:37.941021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:40111 (local address 127.7.24.254:42011)
0620 14:53:37.941145 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:37.941150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:37.941164 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:37.941257 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:37.941260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:37.941302 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:37.941373 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:37.941683 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.942475 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.943199 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:37.943535 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:37.943634 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:37.946063 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:40111: BlockingRecv error: recv got EOF from 127.7.24.193:40111 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260620 14:53:37.946265 16159 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 14:53:38.038146 16163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:38.038398 16163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:38.038504 16163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:38.042443 16163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:38.042512 16163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:38.042569 16163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:38.047762 16163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.194.keytab
--principal=oryx/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:35721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=40187
--tserver_master_addrs=127.7.24.254:42011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:38.048840 16163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:38.049890 16163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:38.059556 16168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:38.059540 16169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:38.059543 16171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:38.059823 16163 server_base.cc:1034] running on GCE node
I20260620 14:53:38.060310 16163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:38.060952 16163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:38.062131 16163 hybrid_clock.cc:648] HybridClock initialized: now 1781967218062096 us; error 44 us; skew 500 ppm
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967218, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:38.065462 16163 init.cc:377] Logged in from keytab as oryx/127.7.24.194@KRBTEST.COM (short username oryx)
I20260620 14:53:38.066366 16163 webserver.cc:460] Webserver started at http://127.7.24.194:40187/ using document root <none> and password file <none>
I20260620 14:53:38.066970 16163 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:38.067056 16163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:38.071250 16163 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:53:38.073320 16178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:38.074687 16163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:38.074790 16163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "6b0391796c544c3e91c3a8d0c1f65ead"
format_stamp: "Formatted at 2026-06-20 14:53:37 on dist-test-slave-sms9"
server_key: "988285621c63db493bb2774d1c62878b"
server_key_iv: "103217c217e9d8638bf9d0aef077b9c0"
server_key_version: "encryptionkey@0"
I20260620 14:53:38.075358 16163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:38.087198 16163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:38.090652 16163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:38.090852 16163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:38.091430 16163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:38.092252 16163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:38.092305 16163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:38.092373 16163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:38.092413 16163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:38.101181 16163 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:35721
I20260620 14:53:38.101256 16291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:35721 every 8 connection(s)
I20260620 14:53:38.102155 16163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:38.109690 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 16163
I20260620 14:53:38.110145 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:37137
--local_ip_for_outbound_sockets=127.7.24.195
--tserver_master_addrs=127.7.24.254:42011
--webserver_port=37241
--webserver_interface=127.7.24.195
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.24.195
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
W20260620 14:53:38.113060 16161 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:38.103930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:34345 (local address 127.7.24.254:42011)
0620 14:53:38.104071 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:38.104075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:38.104878 (+ 803us) server_negotiation.cc:407] Connection header received
0620 14:53:38.106186 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:38.106189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:38.106259 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:38.106364 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:38.107537 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.108074 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.109213 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.109431 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.109554 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.112938 (+ 3384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:34345: BlockingRecv error: recv got EOF from 127.7.24.194:34345 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 14:53:38.113651 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:38.104218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:34345)
0620 14:53:38.104701 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:38.104756 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:38.105967 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:38.106561 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:38.106568 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:38.106910 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:38.107367 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.107380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.108217 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.108221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.109087 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.109096 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.109583 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.109587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.109790 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.110428 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:38.110444 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:38.112820 (+ 2376us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":134,"threads_started":1}
W20260620 14:53:38.114145 16292 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:42011 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.24.254:42011: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:38.120210 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:38.114700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:58269)
0620 14:53:38.114792 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:38.114807 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:38.114886 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:38.115120 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:38.115122 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:38.115255 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:38.115343 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.115348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.116447 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.116450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.116988 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.116996 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.117688 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.117691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.117768 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.118301 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:38.118312 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:38.120117 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260620 14:53:38.120471 16161 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:38.114773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:58269 (local address 127.7.24.254:42011)
0620 14:53:38.114873 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:38.114877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:38.114891 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:38.114951 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:38.114954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:38.114989 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:38.115049 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:38.115468 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.116317 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.117104 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.117510 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.117639 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.120299 (+ 2660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:58269: BlockingRecv error: recv got EOF from 127.7.24.194:58269 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:38.126008 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:38.120871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:51221)
0620 14:53:38.120961 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:38.120976 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:38.121058 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:38.121298 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:38.121301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:38.121448 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:38.121522 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.121527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.122242 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.122248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.123064 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.123079 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.123764 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.123767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.123825 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.124280 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:38.124291 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:38.125920 (+ 1629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260620 14:53:38.126264 16161 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:38.120960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:51221 (local address 127.7.24.254:42011)
0620 14:53:38.121098 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:38.121105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:38.121120 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:53:38.121155 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:38.121157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:38.121202 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:38.121284 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:38.121633 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.122139 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.123192 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.123650 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.123755 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.126065 (+ 2310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:51221: BlockingRecv error: recv got EOF from 127.7.24.194:51221 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260620 14:53:38.126339 16292 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 14:53:38.223941 16295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:38.224207 16295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:38.224251 16295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:38.228176 16295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:38.228231 16295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:38.228291 16295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:38.233418 16295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/krb5kdc/oryx_127.7.24.195.keytab
--principal=oryx/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:37137
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=37241
--tserver_master_addrs=127.7.24.254:42011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:38.234480 16295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:38.235447 16295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:38.245242 16303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:38.245216 16300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:38.245393 16295 server_base.cc:1034] running on GCE node
W20260620 14:53:38.245213 16301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:38.245910 16295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:38.246628 16295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:38.247828 16295 hybrid_clock.cc:648] HybridClock initialized: now 1781967218247807 us; error 32 us; skew 500 ppm
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967218, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:38.250825 16295 init.cc:377] Logged in from keytab as oryx/127.7.24.195@KRBTEST.COM (short username oryx)
I20260620 14:53:38.251752 16295 webserver.cc:460] Webserver started at http://127.7.24.195:37241/ using document root <none> and password file <none>
I20260620 14:53:38.252286 16295 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:38.252359 16295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:38.256346 16295 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:53:38.258538 16310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:38.259990 16295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:53:38.260125 16295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "cd4bde19722645bdb09531b5491285cb"
format_stamp: "Formatted at 2026-06-20 14:53:37 on dist-test-slave-sms9"
server_key: "a94beaa204796ae19e3def9a351aa58d"
server_key_iv: "82d208586e99ec04765323619978159c"
server_key_version: "encryptionkey@0"
I20260620 14:53:38.260641 16295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:38.281050 16295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:38.284093 16295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:38.284256 16295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:38.284788 16295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:38.285563 16295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:38.285611 16295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:38.285652 16295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:38.285668 16295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:38.295114 16295 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:37137
I20260620 14:53:38.295140 16423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:37137 every 8 connection(s)
I20260620 14:53:38.296125 16295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:38.299373 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 16295
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
W20260620 14:53:38.307168 16161 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:38.297963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:41579 (local address 127.7.24.254:42011)
0620 14:53:38.298103 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:38.298107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:38.298946 (+ 839us) server_negotiation.cc:407] Connection header received
0620 14:53:38.300462 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:38.300466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:38.300520 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:38.300633 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:38.301769 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.302267 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.303438 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.303654 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.303761 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.307059 (+ 3298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:41579: BlockingRecv error: recv got EOF from 127.7.24.195:41579 (error 108)
Metrics: {"server-negotiator.queue_time_us":54,"spinlock_wait_cycles":48896}
I20260620 14:53:38.307925 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:38.298319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:41579)
0620 14:53:38.298799 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:38.298841 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:38.300237 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:38.300809 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:38.300816 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:38.301162 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:38.301635 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.301648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.302390 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.302394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.303317 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.303327 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.303797 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.303802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.303962 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.304507 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:38.304522 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:38.306948 (+ 2426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":137,"threads_started":1}
W20260620 14:53:38.308449 16424 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:42011 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.24.254:42011: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
I20260620 14:53:38.309474 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:38.300389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35110 (local address 127.7.24.254:42011)
0620 14:53:38.300622 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:38.300625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:38.300637 (+ 12us) server_negotiation.cc:407] Connection header received
0620 14:53:38.300667 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:38.300669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:38.300732 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:38.300832 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:38.301553 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.302080 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.302992 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.303215 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.303332 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.304896 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:38.304902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:38.304917 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:38.304942 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:38.306817 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:38.307485 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:38.307499 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:38.307504 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:38.307630 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:38.307979 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:38.307981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:38.307983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:38.308507 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:38.308712 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:38.309099 (+ 387us) server_negotiation.cc:299] Negotiation successful
0620 14:53:38.309354 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"spinlock_wait_cycles":384,"thread_start_us":102,"threads_started":1}
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:38.315493 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:38.308942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:56159)
0620 14:53:38.309680 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:38.309695 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:38.309786 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:38.310027 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:38.310030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:38.310196 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:38.310317 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.310324 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.311373 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.311380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.312093 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.312102 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.312634 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.312637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.312705 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.313240 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:38.313252 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:38.315369 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":665}
W20260620 14:53:38.315634 16161 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:38.308995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:56159 (local address 127.7.24.254:42011)
0620 14:53:38.309111 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:38.309114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:38.309770 (+ 656us) server_negotiation.cc:407] Connection header received
0620 14:53:38.309874 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:38.309877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:38.309916 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:38.309997 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:38.310451 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.311243 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.312203 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.312518 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.312612 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.315544 (+ 2932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:56159: BlockingRecv error: recv got EOF from 127.7.24.195:56159 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:38.321244 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:38.316241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:38991)
0620 14:53:38.316380 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:38.316393 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:38.316493 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:38.316719 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:38.316722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:38.316890 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:38.316961 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.316966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.317673 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.317679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.318194 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.318201 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.318724 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.318728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.318810 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.319272 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:38.319284 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:38.321131 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":71}
W20260620 14:53:38.321491 16424 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 14:53:38.321602 16161 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:38.316346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:38991 (local address 127.7.24.254:42011)
0620 14:53:38.316506 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:38.316510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:38.316525 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:53:38.316570 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:38.316573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:38.316618 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:38.316704 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:38.317071 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.317566 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.318318 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.318612 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.318732 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.321483 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:38991: BlockingRecv error: recv got EOF from 127.7.24.195:38991 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:38 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:38.953544 16429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:38.946993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:39065)
0620 14:53:38.947350 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:38.947364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:38.947458 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:38.947728 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:38.947730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:38.947912 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:38.948080 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.948088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.949170 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.949174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.949976 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:38.949986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.950628 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.950632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:38.950730 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.951420 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:38.951433 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:38.953368 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":99,"threads_started":1}
W20260620 14:53:38.953625 16430 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:38.947136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:39065 (local address 127.7.24.254:42011)
0620 14:53:38.947419 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:38.947422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:38.947437 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:53:38.947560 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:38.947562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:38.947607 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:38.947687 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:38.948229 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.949001 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.950112 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:38.950350 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:38.950443 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:38.953500 (+ 3057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:39065: BlockingRecv error: recv got EOF from 127.7.24.193:39065 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":92,"threads_started":1}
Jun 20 14:53:39 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:39 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:39.132263 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:39.127010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:35531)
0620 14:53:39.127274 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:39.127295 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:39.127396 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:39.127626 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:39.127628 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:39.127825 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:39.127919 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:39.127924 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.128671 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.128673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:39.129259 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:39.129268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.129749 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.129751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:39.129812 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:39.130297 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:39.130309 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:39.132138 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":98,"threads_started":1}
W20260620 14:53:39.132472 16430 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:39.127115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:35531 (local address 127.7.24.254:42011)
0620 14:53:39.127258 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:39.127261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:39.127393 (+ 132us) server_negotiation.cc:407] Connection header received
0620 14:53:39.127459 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:39.127462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:39.127510 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:39.127589 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:39.128031 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.128565 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.129389 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.129614 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.129706 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:39.132316 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:35531: BlockingRecv error: recv got EOF from 127.7.24.194:35531 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 20 14:53:39 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:39 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:39.327768 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:39.322170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:50891)
0620 14:53:39.322523 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:39.322538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:39.322642 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:39.322869 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:39.322871 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:39.323049 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:39.323150 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:39.323155 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.323949 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.323951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:39.324527 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:39.324535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.324995 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.324998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:39.325057 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:39.325566 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:39.325577 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:39.327576 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":108,"threads_started":1}
W20260620 14:53:39.327855 16430 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:39.322231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:50891 (local address 127.7.24.254:42011)
0620 14:53:39.322378 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:39.322383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:39.322618 (+ 235us) server_negotiation.cc:407] Connection header received
0620 14:53:39.322713 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:39.322715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:39.322762 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:39.322832 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:39.323281 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.323845 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.324669 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.324892 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.324971 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:39.327718 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:50891: BlockingRecv error: recv got EOF from 127.7.24.195:50891 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 20 14:53:39 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:39 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:39.961951 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:39.954770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:48297)
0620 14:53:39.955129 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:39.955155 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:39.955299 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:39.955750 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:39.955753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:39.956061 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:39.956212 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:39.956220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.957304 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.957311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:39.958101 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:39.958110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.958737 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.958742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:39.958829 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:39.959562 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:39.959575 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:39.961792 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":121,"threads_started":1}
W20260620 14:53:39.962180 16434 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:39.954856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:48297 (local address 127.7.24.254:42011)
0620 14:53:39.955135 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:39.955147 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:39.955273 (+ 126us) server_negotiation.cc:407] Connection header received
0620 14:53:39.955486 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:39.955491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:39.955581 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:39.955710 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:39.956344 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.957180 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.958227 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:39.958609 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:39.958749 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:39.962012 (+ 3263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:48297: BlockingRecv error: recv got EOF from 127.7.24.193:48297 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":77,"threads_started":1}
Jun 20 14:53:40 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:40 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:40.140424 16435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:40.133373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:59381)
0620 14:53:40.133761 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:40.133779 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:40.133911 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:40.134168 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:40.134170 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:40.134406 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:40.134616 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:40.134624 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.135566 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.135570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:40.136247 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:40.136257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.136906 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.136910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:40.137002 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:40.137727 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:40.137740 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:40.140169 (+ 2429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":154,"threads_started":1}
W20260620 14:53:40.140468 16434 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:40.133450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:59381 (local address 127.7.24.254:42011)
0620 14:53:40.133606 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:40.133610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:40.133869 (+ 259us) server_negotiation.cc:407] Connection header received
0620 14:53:40.133993 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:40.133995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:40.134047 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:40.134126 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:40.134820 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.135425 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.136400 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.136688 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.136904 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:40.140339 (+ 3435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:59381: BlockingRecv error: recv got EOF from 127.7.24.194:59381 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 20 14:53:40 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:40 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:40.335078 16436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:40.328826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:51397)
0620 14:53:40.329211 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:40.329228 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:40.329372 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:40.329634 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:40.329636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:40.329881 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:40.330005 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:40.330011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.330864 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.330868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:40.331492 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:40.331503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.332021 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.332025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:40.332123 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:40.332736 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:40.332748 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:40.334854 (+ 2106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":114,"threads_started":1}
W20260620 14:53:40.335129 16434 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:40.329050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:51397 (local address 127.7.24.254:42011)
0620 14:53:40.329222 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:40.329227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:40.329303 (+ 76us) server_negotiation.cc:407] Connection header received
0620 14:53:40.329456 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:40.329459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:40.329512 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:40.329583 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:40.330128 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.330744 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.331641 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.331895 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.332019 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:40.335008 (+ 2989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:51397: BlockingRecv error: recv got EOF from 127.7.24.195:51397 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 20 14:53:40 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:40 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:40.969734 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:40.962897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:47987)
0620 14:53:40.963199 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:40.963214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:40.963308 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:40.963723 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:40.963729 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:40.963941 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:40.964108 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:40.964117 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.965366 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.965369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:40.966141 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:40.966153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.966685 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.966688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:40.966755 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:40.967362 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:40.967375 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:40.969588 (+ 2213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":103,"threads_started":1}
W20260620 14:53:40.969967 16438 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:40.963053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:47987 (local address 127.7.24.254:42011)
0620 14:53:40.963350 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:40.963354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:40.963370 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:53:40.963455 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:40.963464 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:40.963519 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:40.963690 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:40.964248 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.965213 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.966272 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:40.966532 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:40.966650 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:40.969801 (+ 3151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:47987: BlockingRecv error: recv got EOF from 127.7.24.193:47987 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":133,"threads_started":1}
Jun 20 14:53:41 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:41 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:41.147584 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:41.141407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:46251)
0620 14:53:41.141761 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:41.141775 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:41.141888 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:41.142184 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:41.142186 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:41.142379 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:41.142573 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:41.142582 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.143510 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.143514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:41.144347 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:41.144357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.144811 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.144814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:41.144874 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:41.145393 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:41.145403 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:41.147401 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":76,"threads_started":1}
W20260620 14:53:41.147620 16438 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:41.141557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:46251 (local address 127.7.24.254:42011)
0620 14:53:41.141746 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:41.141749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:41.141868 (+ 119us) server_negotiation.cc:407] Connection header received
0620 14:53:41.141975 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:41.141977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:41.142023 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:41.142103 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:41.142725 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.143386 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.144456 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.144705 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.144788 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:41.147524 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:46251: BlockingRecv error: recv got EOF from 127.7.24.194:46251 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Jun 20 14:53:41 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:41 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:41.341989 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:41.335974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:50761)
0620 14:53:41.336307 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:41.336321 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:41.336465 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:41.336809 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:41.336812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:41.337034 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:41.337147 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:41.337152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.337917 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.337921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:41.338576 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:41.338585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.339063 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.339067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:41.339189 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:41.339790 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:41.339802 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:41.341858 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":89,"threads_started":1}
W20260620 14:53:41.342271 16438 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:41.336129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:50761 (local address 127.7.24.254:42011)
0620 14:53:41.336259 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:41.336263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:41.336450 (+ 187us) server_negotiation.cc:407] Connection header received
0620 14:53:41.336635 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:41.336638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:41.336688 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:41.336782 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:41.337262 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.337798 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.338698 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.338947 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.339061 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:41.342156 (+ 3095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:50761: BlockingRecv error: recv got EOF from 127.7.24.195:50761 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 20 14:53:41 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:41 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:41.981395 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:41.970622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:52521)
0620 14:53:41.970935 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:41.970951 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:41.971051 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:41.971353 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:41.971356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:41.971537 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:41.971692 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:41.971698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.972450 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.972453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:41.973047 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:41.973055 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.973460 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.973462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:41.973520 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:41.979217 (+ 5697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:41.979231 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:41.981274 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":83,"threads_started":1}
W20260620 14:53:41.981598 16442 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:41.970683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:52521 (local address 127.7.24.254:42011)
0620 14:53:41.970977 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:41.970980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:41.970998 (+ 18us) server_negotiation.cc:407] Connection header received
0620 14:53:41.971150 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:41.971152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:41.971200 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:41.971284 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:41.971818 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.972351 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.973154 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:41.973365 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:41.973441 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:41.981417 (+ 7976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:52521: BlockingRecv error: recv got EOF from 127.7.24.193:52521 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":118,"threads_started":1}
Jun 20 14:53:42 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:42 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:42.154752 16443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:42.148433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:56805)
0620 14:53:42.148781 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:42.148798 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:42.148914 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:42.149198 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:42.149201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:42.149379 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:42.149481 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:42.149487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.150263 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.150266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:42.151040 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:42.151050 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.151632 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.151637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:42.151751 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:42.152411 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:42.152424 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:42.154625 (+ 2201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":121,"threads_started":1}
W20260620 14:53:42.154917 16442 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:42.148624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:56805 (local address 127.7.24.254:42011)
0620 14:53:42.148799 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:42.148803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:42.148906 (+ 103us) server_negotiation.cc:407] Connection header received
0620 14:53:42.148987 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:42.148990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:42.149039 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:42.149141 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:42.149617 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.150145 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.151168 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.151444 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.151556 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:42.154751 (+ 3195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:56805: BlockingRecv error: recv got EOF from 127.7.24.194:56805 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Jun 20 14:53:42 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:42 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:42.348387 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:42.342921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:54113)
0620 14:53:42.343247 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:42.343259 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:42.343351 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:42.343622 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:42.343625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:42.343880 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:42.343997 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:42.344003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.344772 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.344775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:42.345343 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:42.345352 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.345843 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.345846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:42.345927 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:42.346428 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:42.346439 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:42.348270 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":115,"threads_started":1}
W20260620 14:53:42.348657 16442 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:42.342966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:54113 (local address 127.7.24.254:42011)
0620 14:53:42.343124 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:42.343129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:42.343341 (+ 212us) server_negotiation.cc:407] Connection header received
0620 14:53:42.343455 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:42.343457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:42.343503 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:42.343576 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:42.344114 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.344643 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.345474 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.345698 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.345785 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:42.348467 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:54113: BlockingRecv error: recv got EOF from 127.7.24.195:54113 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 14:53:42 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:42 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:42.989142 16445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:42.982289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:51827)
0620 14:53:42.982725 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:42.982745 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:42.982871 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:42.983174 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:42.983176 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:42.983383 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:42.983485 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:42.983491 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.984510 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.984517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:42.985505 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:42.985517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.986019 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.986022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:42.986088 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:42.986747 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:42.986760 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:42.988926 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":165,"threads_started":1}
W20260620 14:53:42.989192 16446 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:42.982447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:51827 (local address 127.7.24.254:42011)
0620 14:53:42.982736 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:42.982743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:42.982865 (+ 122us) server_negotiation.cc:407] Connection header received
0620 14:53:42.982982 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:42.982986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:42.983042 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:42.983131 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:42.983688 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.984366 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.985664 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:42.985899 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:42.985988 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:42.989060 (+ 3072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:51827: BlockingRecv error: recv got EOF from 127.7.24.193:51827 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":56,"threads_started":1}
Jun 20 14:53:43 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:43 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:43.161743 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:43.155783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:43649)
0620 14:53:43.156164 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:43.156182 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:43.156297 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:43.156587 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:43.156590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:43.156823 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:43.156914 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:43.156920 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.157705 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.157709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:43.158356 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:43.158364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.158893 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.158897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:43.158980 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:43.159544 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:43.159562 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:43.161569 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":144,"threads_started":1}
W20260620 14:53:43.161805 16446 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:43.156009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:43649 (local address 127.7.24.254:42011)
0620 14:53:43.156166 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:43.156173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:43.156244 (+ 71us) server_negotiation.cc:407] Connection header received
0620 14:53:43.156417 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:43.156421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:43.156477 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:43.156566 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:43.157037 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.157574 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.158493 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.158755 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.158885 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:43.161699 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:43649: BlockingRecv error: recv got EOF from 127.7.24.194:43649 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 20 14:53:43 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:43 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:43.355383 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:43.349320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:38859)
0620 14:53:43.349604 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:43.349626 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:43.349747 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:43.350070 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:43.350073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:43.350324 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:43.350450 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:43.350457 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.351521 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.351525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:43.352127 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:43.352136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.352587 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.352590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:43.352652 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:43.353256 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:43.353272 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:43.355263 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
W20260620 14:53:43.355546 16446 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:43.349458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:38859 (local address 127.7.24.254:42011)
0620 14:53:43.349656 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:43.349659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:43.349701 (+ 42us) server_negotiation.cc:407] Connection header received
0620 14:53:43.349869 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:43.349873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:43.349939 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:43.350024 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:43.350670 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.351407 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.352238 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.352444 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.352524 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:43.355408 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:38859: BlockingRecv error: recv got EOF from 127.7.24.195:38859 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Jun 20 14:53:43 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:43 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:43.995850 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:43.989971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:47961)
0620 14:53:43.990242 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:43.990256 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:43.990348 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:43.990699 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:43.990701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:43.990882 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:43.990977 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:43.990983 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.991793 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.991796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:43.992350 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:43.992359 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.992846 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.992849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:43.992919 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:43.993503 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:43.993515 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:43.995675 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":105,"threads_started":1}
W20260620 14:53:43.995988 16450 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:43.990066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:47961 (local address 127.7.24.254:42011)
0620 14:53:43.990290 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:43.990294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:43.990309 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:53:43.990435 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:43.990437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:43.990504 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:43.990591 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:43.991114 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.991664 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.992483 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:43.992714 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:43.992805 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:43.995830 (+ 3025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:47961: BlockingRecv error: recv got EOF from 127.7.24.193:47961 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":83,"threads_started":1}
Jun 20 14:53:44 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:44 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:44.168517 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:44.162521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:37423)
0620 14:53:44.162855 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:44.162876 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:44.163015 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:44.163306 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:44.163309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:44.163528 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:44.163741 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:44.163748 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.164514 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.164520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:44.165176 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:44.165186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.165640 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.165644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:44.165730 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:44.166239 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:44.166250 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:44.168386 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":88,"threads_started":1}
W20260620 14:53:44.168767 16450 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:44.162636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:37423 (local address 127.7.24.254:42011)
0620 14:53:44.162814 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:44.162819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:44.162952 (+ 133us) server_negotiation.cc:407] Connection header received
0620 14:53:44.163145 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:44.163153 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:44.163198 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:44.163285 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:44.163869 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.164396 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.165300 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.165526 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.165637 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:44.168653 (+ 3016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:37423: BlockingRecv error: recv got EOF from 127.7.24.194:37423 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 20 14:53:44 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:44 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:44.362079 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:44.356361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:47513)
0620 14:53:44.356642 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:44.356656 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:44.356755 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:44.356984 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:44.356986 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:44.357182 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:44.357278 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:44.357283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.358123 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.358125 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:44.358864 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:44.358875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.359334 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.359336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:44.359396 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:44.359920 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:44.359933 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:44.361904 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":95,"threads_started":1}
W20260620 14:53:44.362202 16450 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:44.356447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:47513 (local address 127.7.24.254:42011)
0620 14:53:44.356639 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:44.356642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:44.356705 (+ 63us) server_negotiation.cc:407] Connection header received
0620 14:53:44.356823 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:44.356825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:44.356870 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:44.356944 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:44.357377 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.358022 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.358961 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.359238 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.359321 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:44.362052 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:47513: BlockingRecv error: recv got EOF from 127.7.24.195:47513 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 20 14:53:44 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:44 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:45.002552 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:44.996644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:52737)
0620 14:53:44.997046 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:44.997060 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:44.997166 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:44.997402 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:44.997404 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:44.997630 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:44.997724 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:44.997729 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.998594 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.998597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:44.999173 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:44.999182 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.999804 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.999807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:44.999875 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:45.000453 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:45.000465 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:45.002320 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":95,"threads_started":1}
W20260620 14:53:45.002844 16454 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:44.996772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:52737 (local address 127.7.24.254:42011)
0620 14:53:44.997029 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:44.997034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:44.997135 (+ 101us) server_negotiation.cc:407] Connection header received
0620 14:53:44.997245 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:44.997247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:44.997292 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:44.997372 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:44.997831 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.998380 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.999281 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:44.999504 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:44.999635 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:45.002527 (+ 2892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:52737: BlockingRecv error: recv got EOF from 127.7.24.193:52737 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":57,"threads_started":1}
Jun 20 14:53:45 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:45 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:45.176538 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:45.169380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:40805)
0620 14:53:45.169721 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:45.169744 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:45.169840 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:45.170183 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:45.170186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:45.170423 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:45.170574 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:45.170582 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:45.171946 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:45.171949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:45.172518 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:45.172526 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:45.173242 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:45.173245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:45.173313 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:45.173911 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:45.173922 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:45.176424 (+ 2502us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":154,"threads_started":1}
W20260620 14:53:45.176789 16454 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:45.169464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:40805 (local address 127.7.24.254:42011)
0620 14:53:45.169709 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:45.169716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:45.169798 (+ 82us) server_negotiation.cc:407] Connection header received
0620 14:53:45.169930 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:45.169936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:45.170000 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:45.170106 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:45.170715 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:45.171791 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:45.172646 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:45.173103 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:45.173222 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:45.176623 (+ 3401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:40805: BlockingRecv error: recv got EOF from 127.7.24.194:40805 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 20 14:53:45 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:45 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:45.369271 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:45.363012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:43631)
0620 14:53:45.363344 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:45.363360 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:45.363465 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:45.363748 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:45.363750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:45.363986 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:45.364082 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:45.364090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:45.365113 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:45.365118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:45.365928 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:45.365941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:45.366441 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:45.366448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:45.366581 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:45.367124 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:45.367135 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:45.369097 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":95,"threads_started":1}
W20260620 14:53:45.369336 16454 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:45.363067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:43631 (local address 127.7.24.254:42011)
0620 14:53:45.363248 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:45.363252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:45.363449 (+ 197us) server_negotiation.cc:407] Connection header received
0620 14:53:45.363543 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:45.363546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:45.363594 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:45.363664 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:45.364176 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:45.364874 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:45.366067 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:45.366340 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:45.366420 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:45.369229 (+ 2809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:43631: BlockingRecv error: recv got EOF from 127.7.24.195:43631 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Jun 20 14:53:46 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:46 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:46.009147 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:46.003445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:34185)
0620 14:53:46.003736 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:46.003751 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:46.003842 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:46.004113 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:46.004115 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:46.004297 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:46.004391 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:46.004396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.005218 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.005221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:46.005804 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:46.005813 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.006367 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.006370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:46.006441 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:46.007033 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:46.007046 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:46.009026 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":114,"threads_started":1}
W20260620 14:53:46.009366 16458 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:46.003657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:34185 (local address 127.7.24.254:42011)
0620 14:53:46.003894 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:46.003898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:46.003910 (+ 12us) server_negotiation.cc:407] Connection header received
0620 14:53:46.003947 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:46.003950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:46.003991 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:46.004075 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:46.004539 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.005077 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.005956 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.006209 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.006299 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:46.009198 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:34185: BlockingRecv error: recv got EOF from 127.7.24.193:34185 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":89,"threads_started":1}
Jun 20 14:53:46 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:46 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:46.182955 16459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:46.177341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:59081)
0620 14:53:46.177634 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:46.177652 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:46.177779 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:46.178048 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:46.178051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:46.178262 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:46.178362 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:46.178368 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.179384 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.179390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:46.179979 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:46.179987 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.180443 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.180447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:46.180533 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:46.181024 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:46.181036 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:46.182833 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":78,"threads_started":1}
W20260620 14:53:46.183246 16458 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:46.177444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:59081 (local address 127.7.24.254:42011)
0620 14:53:46.177591 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:46.177595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:46.177739 (+ 144us) server_negotiation.cc:407] Connection header received
0620 14:53:46.177875 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:46.177878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:46.177925 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:46.178021 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:46.178502 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.179280 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.180109 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.180330 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.180440 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:46.182982 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:59081: BlockingRecv error: recv got EOF from 127.7.24.194:59081 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 20 14:53:46 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:46 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:46.375465 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:46.370117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:42315)
0620 14:53:46.370374 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:46.370389 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:46.370549 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:46.370792 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:46.370794 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:46.370983 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:46.371079 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:46.371101 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.371789 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.371791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:46.372350 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:46.372358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.372815 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.372818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:46.372882 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:46.373422 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:46.373434 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:46.375348 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":94,"threads_started":1}
W20260620 14:53:46.375738 16458 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:46.370215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:42315 (local address 127.7.24.254:42011)
0620 14:53:46.370347 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:46.370351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:46.370489 (+ 138us) server_negotiation.cc:407] Connection header received
0620 14:53:46.370640 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:46.370642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:46.370690 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:46.370759 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:46.371189 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.371697 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.372462 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:46.372708 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:46.372787 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:46.375492 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:42315: BlockingRecv error: recv got EOF from 127.7.24.195:42315 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 20 14:53:47 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:47 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:47.020573 16461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:47.010035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:43105)
0620 14:53:47.010347 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:47.010362 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:47.010506 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:47.010763 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:47.010765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:47.010950 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:47.011047 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:47.011052 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.011782 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.011784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:47.012352 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:47.012360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.012808 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.012811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:47.012871 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:47.018348 (+ 5477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:47.018361 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:47.020451 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":134,"threads_started":1}
W20260620 14:53:47.020820 16462 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:47.010115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:43105 (local address 127.7.24.254:42011)
0620 14:53:47.010378 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:47.010382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:47.010414 (+ 32us) server_negotiation.cc:407] Connection header received
0620 14:53:47.010619 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:47.010621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:47.010667 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:47.010749 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:47.011145 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.011684 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.012460 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.012690 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.012771 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:47.020680 (+ 7909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:43105: BlockingRecv error: recv got EOF from 127.7.24.193:43105 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":105,"threads_started":1}
Jun 20 14:53:47 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:47 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:47.189558 16463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:47.183823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:37195)
0620 14:53:47.184195 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:47.184208 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:47.184308 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:47.184604 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:47.184607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:47.184789 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:47.184882 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:47.184888 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.185644 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.185646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:47.186221 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:47.186228 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.186884 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.186890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:47.186973 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:47.187488 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:47.187503 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:47.189405 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":122,"threads_started":1}
W20260620 14:53:47.189721 16462 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:47.183852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:37195 (local address 127.7.24.254:42011)
0620 14:53:47.184011 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:47.184015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:47.184283 (+ 268us) server_negotiation.cc:407] Connection header received
0620 14:53:47.184387 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:47.184390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:47.184436 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:47.184516 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:47.184988 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.185527 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.186348 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.186682 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.186823 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:47.189585 (+ 2762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:37195: BlockingRecv error: recv got EOF from 127.7.24.194:37195 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 20 14:53:47 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:47 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:47.382040 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:47.376230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:42901)
0620 14:53:47.376566 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:47.376580 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:47.376720 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:47.376989 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:47.376992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:47.377203 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:47.377290 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:47.377295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.378062 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.378068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:47.378755 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:47.378764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.379222 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.379225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:47.379313 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:47.379902 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:47.379914 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:47.381848 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":83,"threads_started":1}
W20260620 14:53:47.382108 16462 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:47.376312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:42901 (local address 127.7.24.254:42011)
0620 14:53:47.376482 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:47.376487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:47.376659 (+ 172us) server_negotiation.cc:407] Connection header received
0620 14:53:47.376832 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:47.376838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:47.376888 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:47.376971 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:47.377407 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.377930 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.378883 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:47.379110 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:47.379220 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:47.381980 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:42901: BlockingRecv error: recv got EOF from 127.7.24.195:42901 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 20 14:53:48 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:48 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:48.027022 16465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:48.021325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:42015)
0620 14:53:48.021626 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:48.021644 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:48.021748 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:48.022062 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:48.022065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:48.022270 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:48.022371 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:48.022376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.023203 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.023207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:48.023772 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:48.023780 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.024192 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.024195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:48.024254 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:48.024805 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:48.024818 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:48.026846 (+ 2028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":103,"threads_started":1}
W20260620 14:53:48.027112 16466 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:48.021391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:42015 (local address 127.7.24.254:42011)
0620 14:53:48.021627 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:48.021632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:48.021711 (+ 79us) server_negotiation.cc:407] Connection header received
0620 14:53:48.021857 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:48.021860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:48.021917 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:48.022007 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:48.022518 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.023097 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.023876 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.024095 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.024172 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:48.026982 (+ 2810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:42015: BlockingRecv error: recv got EOF from 127.7.24.193:42015 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":64,"threads_started":1}
Jun 20 14:53:48 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:48 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
W20260620 14:53:48.196128 16466 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:48.190556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:57431 (local address 127.7.24.254:42011)
0620 14:53:48.190722 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:48.190728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:48.190906 (+ 178us) server_negotiation.cc:407] Connection header received
0620 14:53:48.191010 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:48.191013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:48.191057 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:48.191127 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:48.191581 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.192134 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.192943 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.193166 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.193250 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:48.196028 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:57431: BlockingRecv error: recv got EOF from 127.7.24.194:57431 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 14:53:48.196081 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:48.190431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:57431)
0620 14:53:48.190814 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:48.190828 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:48.190929 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:48.191191 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:48.191194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:48.191372 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:48.191468 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:48.191473 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.192231 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.192233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:48.192814 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:48.192822 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.193300 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.193303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:48.193369 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:48.193872 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:48.193883 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:48.195909 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":104,"threads_started":1}
Jun 20 14:53:48 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:48 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:48.388926 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:48.382940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:34101)
0620 14:53:48.383279 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:48.383297 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:48.383443 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:48.383775 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:48.383778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:48.383987 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:48.384081 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:48.384087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.384852 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.384858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:48.385549 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:48.385558 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.386226 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.386229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:48.386308 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:48.386998 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:48.387014 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:48.388811 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":102,"threads_started":1}
W20260620 14:53:48.389143 16466 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:48.383147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:34101 (local address 127.7.24.254:42011)
0620 14:53:48.383302 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:48.383308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:48.383384 (+ 76us) server_negotiation.cc:407] Connection header received
0620 14:53:48.383606 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:48.383609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:48.383667 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:48.383757 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:48.384204 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.384739 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.385715 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:48.386012 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:48.386155 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:48.389006 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:34101: BlockingRecv error: recv got EOF from 127.7.24.195:34101 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Jun 20 14:53:49 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:49 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:49.033778 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:49.027900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:33387)
0620 14:53:49.028224 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:49.028240 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:49.028336 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:49.028839 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:49.028841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:49.029022 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:49.029122 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:49.029127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.029854 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.029857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:49.030417 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:49.030424 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.030954 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.030957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:49.031045 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:49.031574 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:49.031587 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:49.033543 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":72,"threads_started":1}
W20260620 14:53:49.033787 16470 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:49.027979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:33387 (local address 127.7.24.254:42011)
0620 14:53:49.028281 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:49.028284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:49.028298 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:49.028421 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:49.028423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:49.028481 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:49.028559 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:49.029227 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.029749 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.030607 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.030835 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.030950 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:49.033669 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:33387: BlockingRecv error: recv got EOF from 127.7.24.193:33387 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":90,"threads_started":1}
Jun 20 14:53:49 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:49 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:49.203325 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:49.197211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:48439)
0620 14:53:49.197586 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:49.197605 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:49.197750 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:49.198024 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:49.198026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:49.198199 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:49.198295 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:49.198301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.199094 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.199096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:49.199646 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:49.199654 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.200187 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.200190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:49.200299 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:49.200982 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:49.200997 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:49.203141 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":85,"threads_started":1}
W20260620 14:53:49.203380 16470 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:49.197385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:48439 (local address 127.7.24.254:42011)
0620 14:53:49.197569 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:49.197574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:49.197693 (+ 119us) server_negotiation.cc:407] Connection header received
0620 14:53:49.197846 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:49.197849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:49.197909 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:49.198004 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:49.198405 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.198975 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.199796 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.200062 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.200182 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:49.203274 (+ 3092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:48439: BlockingRecv error: recv got EOF from 127.7.24.194:48439 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Jun 20 14:53:49 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:49 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:49.395788 16472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:49.389797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:39307)
0620 14:53:49.390088 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:49.390102 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:49.390191 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:49.390510 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:49.390514 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:49.390774 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:49.390912 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:49.390919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.392008 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.392011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:49.392641 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:49.392651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.393099 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.393101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:49.393162 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:49.393730 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:49.393752 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:49.395668 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":107,"threads_started":1}
W20260620 14:53:49.395998 16470 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:49.389991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:39307 (local address 127.7.24.254:42011)
0620 14:53:49.390132 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:49.390136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:49.390155 (+ 19us) server_negotiation.cc:407] Connection header received
0620 14:53:49.390285 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:49.390288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:49.390335 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:49.390418 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:49.391168 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.391895 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.392775 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:49.392994 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:49.393084 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:49.395855 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:39307: BlockingRecv error: recv got EOF from 127.7.24.195:39307 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 14:53:50 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:50 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:50.040263 16473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:50.034698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:52893)
0620 14:53:50.034978 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:50.034992 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:50.035088 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:50.035334 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:50.035336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:50.035527 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:50.035640 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:50.035646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.036369 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.036372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:50.036933 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:50.036940 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.037344 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.037346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:50.037404 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:50.038087 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:50.038104 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:50.040146 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":97,"threads_started":1}
W20260620 14:53:50.040412 16474 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:50.034764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:52893 (local address 127.7.24.254:42011)
0620 14:53:50.034998 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:50.035002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:50.035040 (+ 38us) server_negotiation.cc:407] Connection header received
0620 14:53:50.035174 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:50.035177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:50.035221 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:50.035300 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:50.035750 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.036271 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.037030 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.037248 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.037323 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:50.040253 (+ 2930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:52893: BlockingRecv error: recv got EOF from 127.7.24.193:52893 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":88,"threads_started":1}
Jun 20 14:53:50 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:50 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:50.210999 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:50.204421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:38301)
0620 14:53:50.204786 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:50.204802 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:50.204954 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:50.205253 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:50.205256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:50.205850 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:50.205960 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:50.205966 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.206842 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.206846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:50.207404 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:50.207412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.207882 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.207886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:50.207971 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:50.208534 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:50.208545 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:50.210872 (+ 2327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":113,"threads_started":1}
W20260620 14:53:50.211561 16474 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:50.204631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:38301 (local address 127.7.24.254:42011)
0620 14:53:50.204773 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:50.204778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:50.204890 (+ 112us) server_negotiation.cc:407] Connection header received
0620 14:53:50.205067 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:50.205072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:50.205127 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:50.205226 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:50.206105 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.206728 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.207528 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.207760 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.207874 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:50.211392 (+ 3518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:38301: BlockingRecv error: recv got EOF from 127.7.24.194:38301 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 20 14:53:50 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:50 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:50.402557 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:50.396644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:53971)
0620 14:53:50.396925 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:50.396938 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:50.397105 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:50.397393 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:50.397396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:50.397610 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:50.397702 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:50.397708 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.398508 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.398514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:50.399106 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:50.399116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.399574 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.399577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:50.399658 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:50.400163 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:50.400175 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:50.402368 (+ 2193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
W20260620 14:53:50.402758 16474 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:50.396714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:53971 (local address 127.7.24.254:42011)
0620 14:53:50.396870 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:50.396874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:50.397025 (+ 151us) server_negotiation.cc:407] Connection header received
0620 14:53:50.397214 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:50.397219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:50.397277 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:50.397366 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:50.397822 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.398348 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.399236 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:50.399459 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:50.399579 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:50.402588 (+ 3009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:53971: BlockingRecv error: recv got EOF from 127.7.24.195:53971 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 14:53:51 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:51 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:51.046813 16477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:51.041087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:59587)
0620 14:53:51.041361 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:51.041378 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:51.041480 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:51.041820 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:51.041823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:51.042008 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:51.042107 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:51.042112 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.042952 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.042955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:51.043508 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:51.043516 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.043961 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.043963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:51.044027 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:51.044562 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:51.044574 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:51.046592 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":100,"threads_started":1}
W20260620 14:53:51.047016 16478 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:51.041157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:59587 (local address 127.7.24.254:42011)
0620 14:53:51.041403 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:51.041406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:51.041434 (+ 28us) server_negotiation.cc:407] Connection header received
0620 14:53:51.041559 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:51.041561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:51.041604 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:51.041716 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:51.042224 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.042840 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.043635 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.043855 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.043932 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:51.046856 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:59587: BlockingRecv error: recv got EOF from 127.7.24.193:59587 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":99,"threads_started":1}
Jun 20 14:53:51 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:51 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:51.217911 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:51.211894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:38665)
0620 14:53:51.212241 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:51.212258 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:51.212415 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:51.212709 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:51.212712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:51.212928 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:51.213019 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:51.213024 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.213783 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.213789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:51.214361 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:51.214369 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.214856 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.214859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:51.214938 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:51.215610 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:51.215629 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:51.217783 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":98,"threads_started":1}
W20260620 14:53:51.218216 16478 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:51.212091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:38665 (local address 127.7.24.254:42011)
0620 14:53:51.212243 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:51.212249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:51.212349 (+ 100us) server_negotiation.cc:407] Connection header received
0620 14:53:51.212520 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:51.212524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:51.212585 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:51.212686 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:51.213147 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.213669 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.214511 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.214734 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.214850 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:51.217939 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:38665: BlockingRecv error: recv got EOF from 127.7.24.194:38665 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 20 14:53:51 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:51 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:51.408881 16480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:51.403364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:47613)
0620 14:53:51.403659 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:51.403673 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:51.403773 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:51.404011 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:51.404013 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:51.404190 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:51.404282 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:51.404286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.405036 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.405039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:51.405624 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:51.405632 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.406105 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.406108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:51.406173 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:51.406762 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:51.406774 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:51.408712 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":95,"threads_started":1}
W20260620 14:53:51.409322 16478 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:51.403465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:47613 (local address 127.7.24.254:42011)
0620 14:53:51.403603 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:51.403606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:51.403774 (+ 168us) server_negotiation.cc:407] Connection header received
0620 14:53:51.403842 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:51.403845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:51.403905 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:51.404003 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:51.404414 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.404929 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.405760 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:51.405979 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:51.406085 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:51.408959 (+ 2874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:47613: BlockingRecv error: recv got EOF from 127.7.24.195:47613 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 20 14:53:52 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:52 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:52.057387 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:52.047632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:58777)
0620 14:53:52.047913 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:52.047932 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:52.048049 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:52.048301 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:52.048303 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:52.048489 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:52.048590 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:52.048595 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.049367 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.049370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:52.049914 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:52.049922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.050446 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.050449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:52.050637 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:52.055283 (+ 4646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:52.055297 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:52.057264 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":75,"threads_started":1}
W20260620 14:53:52.057693 16482 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:52.047772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:58777 (local address 127.7.24.254:42011)
0620 14:53:52.048082 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:52.048086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:52.048101 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:53:52.048143 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:52.048146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:52.048187 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:52.048265 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:52.048716 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.049245 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.050064 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.050306 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.050392 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:52.057483 (+ 7091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:58777: BlockingRecv error: recv got EOF from 127.7.24.193:58777 (error 108)
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":79,"threads_started":1}
Jun 20 14:53:52 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:52 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:52.224335 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:52.218856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:36441)
0620 14:53:52.219219 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:52.219234 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:52.219333 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:52.219573 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:52.219576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:52.219773 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:52.219871 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:52.219876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.220628 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.220631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:52.221243 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:52.221253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.221694 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.221697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:52.221763 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:52.222379 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:52.222391 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:52.224204 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":111,"threads_started":1}
W20260620 14:53:52.224530 16482 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:52.218986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:36441 (local address 127.7.24.254:42011)
0620 14:53:52.219120 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:52.219124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:52.219303 (+ 179us) server_negotiation.cc:407] Connection header received
0620 14:53:52.219412 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:52.219415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:52.219457 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:52.219525 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:52.219971 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.220493 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.221355 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.221564 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.221641 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:52.224360 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:36441: BlockingRecv error: recv got EOF from 127.7.24.194:36441 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 20 14:53:52 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:52 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.195@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:52.416718 16484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:52.410063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.195:54893)
0620 14:53:52.410505 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:52.410518 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:52.410692 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:52.410974 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:52.410977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:52.411198 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:52.411287 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:52.411292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.412061 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.412068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:52.412895 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:52.412908 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.413562 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.413566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:52.413683 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:52.414319 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:52.414335 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:52.416579 (+ 2244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":87,"threads_started":1}
W20260620 14:53:52.416860 16482 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:52.410159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:54893 (local address 127.7.24.254:42011)
0620 14:53:52.410415 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:52.410420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:52.410640 (+ 220us) server_negotiation.cc:407] Connection header received
0620 14:53:52.410802 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:52.410807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:52.410857 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:52.410953 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:52.411405 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.411938 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.413020 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:52.413426 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:52.413532 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:52.416730 (+ 3198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.195:54893: BlockingRecv error: recv got EOF from 127.7.24.195:54893 (error 108)
Metrics: {"server-negotiator.queue_time_us":166}
Jun 20 14:53:53 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:53 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.193@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:53.064416 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:53.058508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.193:43715)
0620 14:53:53.058914 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:53.058929 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:53.059049 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:53.059344 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:53.059347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:53.059604 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:53.059725 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:53.059740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.060554 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.060559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:53.061132 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:53.061139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.061629 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.061632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:53.061709 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:53.062252 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:53.062264 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:53.064294 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":149,"threads_started":1}
W20260620 14:53:53.064702 16486 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:53.058560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:43715 (local address 127.7.24.254:42011)
0620 14:53:53.058835 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:53.058841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:53.059017 (+ 176us) server_negotiation.cc:407] Connection header received
0620 14:53:53.059155 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:53.059158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:53.059204 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:53.059322 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:53.059872 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.060435 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.061264 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.061503 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.061633 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:53.064436 (+ 2803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.193:43715: BlockingRecv error: recv got EOF from 127.7.24.193:43715 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":68,"threads_started":1}
Jun 20 14:53:53 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 14:53:53 dist-test-slave-sms9 krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.24.194@KRBTEST.COM for oryx/127.7.24.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 14:53:53.230948 16487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:53.225298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42011 (local address 127.7.24.194:55347)
0620 14:53:53.225556 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:53.225571 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:53.225654 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:53.225977 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:53.225980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:53.226168 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:53.226260 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:53.226265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.227108 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.227111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:53.227698 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:53.227706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.228139 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.228142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:53.228204 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:53.228815 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:53.228826 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:53.230819 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.24.254:42011: Server oryx/127.7.24.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":66,"threads_started":1}
W20260620 14:53:53.231133 16486 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:53.225383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:55347 (local address 127.7.24.254:42011)
0620 14:53:53.225592 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:53.225596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:53.225614 (+ 18us) server_negotiation.cc:407] Connection header received
0620 14:53:53.225776 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:53.225779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:53.225824 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:53.225903 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:53.226393 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.226994 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.227811 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.228030 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.228112 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:53.230991 (+ 2879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.24.194:55347: BlockingRecv error: recv got EOF from 127.7.24.194:55347 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 14:53:53.308079 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 16011
I20260620 14:53:53.314018 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 16163
I20260620 14:53:53.319790 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 16295
I20260620 14:53:53.325271 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 15941
2026-06-20T14:53:53Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18079 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-20T14:53:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:53Z Disabled control of system clock
I20260620 14:53:53.356600 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40205
--webserver_interface=127.7.24.254
--webserver_port=0
--builtin_ntp_servers=127.7.24.212:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:40205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 14:53:53.465927 16491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:53.466197 16491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:53.466231 16491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:53.470220 16491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:53.470288 16491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:53.470304 16491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:53.470321 16491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:53.470337 16491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:53.475157 16491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:40205
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:53.476197 16491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:53.477494 16491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:53.485646 16496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:53.485648 16497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:53.485803 16491 server_base.cc:1034] running on GCE node
W20260620 14:53:53.486001 16499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:53.486588 16491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:53.487629 16491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:53.488787 16491 hybrid_clock.cc:648] HybridClock initialized: now 1781967233488770 us; error 30 us; skew 500 ppm
I20260620 14:53:53.490706 16491 webserver.cc:460] Webserver started at http://127.7.24.254:40737/ using document root <none> and password file <none>
I20260620 14:53:53.491375 16491 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:53.491439 16491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:53.491659 16491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:53.493603 16491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "967a8b13807845dd99446aa8444e9c13"
format_stamp: "Formatted at 2026-06-20 14:53:53 on dist-test-slave-sms9"
server_key: "4a021b3c32947e684ae223651402b49a"
server_key_iv: "cb1a84b9a89b7d3c5e2ef43bedb9c1e3"
server_key_version: "encryptionkey@0"
I20260620 14:53:53.494129 16491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "967a8b13807845dd99446aa8444e9c13"
format_stamp: "Formatted at 2026-06-20 14:53:53 on dist-test-slave-sms9"
server_key: "4a021b3c32947e684ae223651402b49a"
server_key_iv: "cb1a84b9a89b7d3c5e2ef43bedb9c1e3"
server_key_version: "encryptionkey@0"
I20260620 14:53:53.499075 16491 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260620 14:53:53.502440 16505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:53.504098 16491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:53:53.504276 16491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "967a8b13807845dd99446aa8444e9c13"
format_stamp: "Formatted at 2026-06-20 14:53:53 on dist-test-slave-sms9"
server_key: "4a021b3c32947e684ae223651402b49a"
server_key_iv: "cb1a84b9a89b7d3c5e2ef43bedb9c1e3"
server_key_version: "encryptionkey@0"
I20260620 14:53:53.504436 16491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:53.517408 16491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:53.518303 16491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:53.518560 16491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:53.525795 16491 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:40205
I20260620 14:53:53.525794 16557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:40205 every 8 connection(s)
I20260620 14:53:53.526912 16491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:53.530182 16558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:53.532320 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 16491
I20260620 14:53:53.532411 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:53.532752 7267 external_mini_cluster.cc:1428] Setting key 6028311618be544260c8094f3e289eb0
I20260620 14:53:53.537837 16558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13: Bootstrap starting.
I20260620 14:53:53.540601 16558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:53.541584 16558 log.cc:824] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:53.543467 16561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:53.534254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53994 (local address 127.7.24.254:40205)
0620 14:53:53.534975 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:53.534998 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:53.535054 (+ 56us) server_negotiation.cc:407] Connection header received
0620 14:53:53.535867 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:53.535893 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:53.536436 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:53.536828 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:53.537283 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.538755 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.539669 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.540347 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.540533 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:53.541384 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:53.541390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:53.541409 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:53.541924 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:53.542003 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:53.542042 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:53.542174 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:53.542316 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 14:53:53.542738 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":156,"threads_started":1}
I20260620 14:53:53.543743 16558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13: No bootstrap required, opened a new log
I20260620 14:53:53.548133 16558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967a8b13807845dd99446aa8444e9c13" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40205 } }
I20260620 14:53:53.548398 16558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:53.548486 16558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 967a8b13807845dd99446aa8444e9c13, State: Initialized, Role: FOLLOWER
I20260620 14:53:53.548911 16558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967a8b13807845dd99446aa8444e9c13" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40205 } }
I20260620 14:53:53.549059 16558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:53.549134 16558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:53.549233 16558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:53.550199 16558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967a8b13807845dd99446aa8444e9c13" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40205 } }
I20260620 14:53:53.550621 16558 leader_election.cc:302] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 967a8b13807845dd99446aa8444e9c13; no voters:
I20260620 14:53:53.550953 16558 leader_election.cc:288] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:53.551151 16563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:53.551470 16563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [term 1 LEADER]: Becoming Leader. State: Replica: 967a8b13807845dd99446aa8444e9c13, State: Running, Role: LEADER
I20260620 14:53:53.551911 16563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967a8b13807845dd99446aa8444e9c13" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40205 } }
I20260620 14:53:53.552177 16558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:53.553681 16565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 967a8b13807845dd99446aa8444e9c13. Latest consensus state: current_term: 1 leader_uuid: "967a8b13807845dd99446aa8444e9c13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967a8b13807845dd99446aa8444e9c13" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40205 } } }
I20260620 14:53:53.553699 16564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "967a8b13807845dd99446aa8444e9c13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967a8b13807845dd99446aa8444e9c13" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40205 } } }
I20260620 14:53:53.553833 16565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:53.553833 16564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13 [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:53.554414 16575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:53.557670 16575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:53.563692 16575 catalog_manager.cc:1347] Generated new cluster ID: 7e2ea0e7a8e34813bbb85d934b2104ac
I20260620 14:53:53.563797 16575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:53.588476 16575 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:53.589498 16575 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:53.595115 16575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 967a8b13807845dd99446aa8444e9c13: Generated new TSK 0
I20260620 14:53:53.595955 16575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:53:53.606819 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40205
--builtin_ntp_servers=127.7.24.212:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 14:53:53.734766 16582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:53.735054 16582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:53.735096 16582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:53.739426 16582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:53.739494 16582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:53.739549 16582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:53.744277 16582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:53.745371 16582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:53.746830 16582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:53.757081 16587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:53.757088 16588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:53.757081 16590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:53.757386 16582 server_base.cc:1034] running on GCE node
I20260620 14:53:53.757941 16582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:53.758716 16582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:53.759922 16582 hybrid_clock.cc:648] HybridClock initialized: now 1781967233759883 us; error 50 us; skew 500 ppm
I20260620 14:53:53.762089 16582 webserver.cc:460] Webserver started at http://127.7.24.193:41095/ using document root <none> and password file <none>
I20260620 14:53:53.762751 16582 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:53.762843 16582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:53.763075 16582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:53.765255 16582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "adf739dba9544181a0c8d478a97c139f"
format_stamp: "Formatted at 2026-06-20 14:53:53 on dist-test-slave-sms9"
server_key: "5fecd4c5a41e8f734db4f23512a497e0"
server_key_iv: "0719645ceaa335feabe964e481630481"
server_key_version: "encryptionkey@0"
I20260620 14:53:53.765834 16582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "adf739dba9544181a0c8d478a97c139f"
format_stamp: "Formatted at 2026-06-20 14:53:53 on dist-test-slave-sms9"
server_key: "5fecd4c5a41e8f734db4f23512a497e0"
server_key_iv: "0719645ceaa335feabe964e481630481"
server_key_version: "encryptionkey@0"
I20260620 14:53:53.769917 16582 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:53:53.772744 16596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:53.774211 16582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 14:53:53.774339 16582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "adf739dba9544181a0c8d478a97c139f"
format_stamp: "Formatted at 2026-06-20 14:53:53 on dist-test-slave-sms9"
server_key: "5fecd4c5a41e8f734db4f23512a497e0"
server_key_iv: "0719645ceaa335feabe964e481630481"
server_key_version: "encryptionkey@0"
I20260620 14:53:53.774442 16582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:53.796764 16582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:53.797468 16582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:53.797641 16582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:53.798188 16582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:53.799168 16582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:53.799216 16582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:53.799256 16582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:53.799273 16582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:53.809180 16582 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:39571
I20260620 14:53:53.809217 16709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:39571 every 8 connection(s)
I20260620 14:53:53.810155 16582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:53.815090 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 16582
I20260620 14:53:53.815238 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:53.815570 7267 external_mini_cluster.cc:1428] Setting key 75c6feef8e34a559679ed81f388ebdca
I20260620 14:53:53.817595 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40205
--builtin_ntp_servers=127.7.24.212:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 14:53:53.820516 16561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:53.812278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:41929 (local address 127.7.24.254:40205)
0620 14:53:53.812422 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:53.812426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:53.813671 (+ 1245us) server_negotiation.cc:407] Connection header received
0620 14:53:53.815165 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:53.815169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:53.815230 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:53.815347 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:53.816311 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.816860 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.818197 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.818432 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.818564 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:53.819962 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:53.819965 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:53.819967 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:53.819980 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:53.820001 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:53.820009 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:53.820080 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:53.820323 (+ 243us) server_negotiation.cc:299] Negotiation successful
0620 14:53:53.820391 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 14:53:53.821411 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:53.812745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40205 (local address 127.7.24.193:41929)
0620 14:53:53.813478 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:53.813520 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:53.814947 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:53.815530 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:53.815537 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:53.815622 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:53.816163 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:53.816177 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.816994 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.816999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:53.818038 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:53.818049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:53.818609 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:53.818613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:53.818829 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:53.819590 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:53.819593 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:53.819658 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:53.819760 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:53.820104 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:53.820116 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:53.820125 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 14:53:53.820264 (+ 139us) client_negotiation.cc:240] Negotiation successful
0620 14:53:53.820516 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":130,"threads_started":1}
I20260620 14:53:53.822820 16710 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40205
I20260620 14:53:53.823071 16710 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:53.823652 16710 heartbeater.cc:511] Master 127.7.24.254:40205 requested a full tablet report, sending...
I20260620 14:53:53.825251 16522 ts_manager.cc:194] Registered new tserver with Master: adf739dba9544181a0c8d478a97c139f (127.7.24.193:39571)
I20260620 14:53:53.826424 16522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.193:41929
W20260620 14:53:53.933516 16713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:53.933785 16713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:53.933822 16713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:53.937933 16713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:53.938001 16713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:53.938055 16713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:53.943116 16713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:53.944204 16713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:53.945586 16713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:53.955637 16721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:53.955657 16719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:53.955691 16713 server_base.cc:1034] running on GCE node
W20260620 14:53:53.955708 16718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:53.956221 16713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:53.956882 16713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:53.958073 16713 hybrid_clock.cc:648] HybridClock initialized: now 1781967233958063 us; error 31 us; skew 500 ppm
I20260620 14:53:53.959874 16713 webserver.cc:460] Webserver started at http://127.7.24.194:38989/ using document root <none> and password file <none>
I20260620 14:53:53.960458 16713 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:53.960552 16713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:53.960786 16713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:53.962909 16713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "c43d3934fb3e4cd68780dcb8abcdf19a"
format_stamp: "Formatted at 2026-06-20 14:53:53 on dist-test-slave-sms9"
server_key: "ae44385c6c0793024fd2f95448b14d14"
server_key_iv: "8196327b24ee4b9bdc87fc292341d06f"
server_key_version: "encryptionkey@0"
I20260620 14:53:53.963476 16713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "c43d3934fb3e4cd68780dcb8abcdf19a"
format_stamp: "Formatted at 2026-06-20 14:53:53 on dist-test-slave-sms9"
server_key: "ae44385c6c0793024fd2f95448b14d14"
server_key_iv: "8196327b24ee4b9bdc87fc292341d06f"
server_key_version: "encryptionkey@0"
I20260620 14:53:53.967161 16713 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:53:53.969691 16727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:53.971128 16713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:53.971273 16713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "c43d3934fb3e4cd68780dcb8abcdf19a"
format_stamp: "Formatted at 2026-06-20 14:53:53 on dist-test-slave-sms9"
server_key: "ae44385c6c0793024fd2f95448b14d14"
server_key_iv: "8196327b24ee4b9bdc87fc292341d06f"
server_key_version: "encryptionkey@0"
I20260620 14:53:53.971397 16713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:53.983916 16713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:53.984623 16713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:53.984827 16713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:53.985401 16713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:53.986334 16713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:53.986408 16713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:53.986500 16713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:53.986555 16713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:53.995280 16713 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:45347
I20260620 14:53:53.995314 16840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:45347 every 8 connection(s)
I20260620 14:53:53.996199 16713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:54.005360 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 16713
I20260620 14:53:54.005533 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:54.005875 7267 external_mini_cluster.cc:1428] Setting key 846e1276462db92865f8d37e629b673e
I20260620 14:53:54.006057 16561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:53.998060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:46121 (local address 127.7.24.254:40205)
0620 14:53:53.998233 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:53.998237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:53.999376 (+ 1139us) server_negotiation.cc:407] Connection header received
0620 14:53:54.000843 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:54.000847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:54.000907 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:54.001025 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:54.001985 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:54.002663 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:54.003824 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:54.004033 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:54.004126 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:54.005389 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:54.005393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:54.005396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:54.005413 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:54.005438 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:54.005449 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:54.005537 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:54.005816 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 14:53:54.005886 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260620 14:53:54.007011 16843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:53.998457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40205 (local address 127.7.24.194:46121)
0620 14:53:53.999219 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:53.999261 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:54.000623 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:54.001204 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:54.001211 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:54.001296 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:54.001806 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:54.001825 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:54.002792 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:54.002795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:54.003702 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:54.003712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:54.004198 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:54.004201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:54.004350 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:54.005085 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:54.005087 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:54.005146 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:54.005213 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:54.005626 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:54.005636 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:54.005643 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 14:53:54.005777 (+ 134us) client_negotiation.cc:240] Negotiation successful
0620 14:53:54.006081 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":124,"threads_started":1}
I20260620 14:53:54.007992 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40205
--builtin_ntp_servers=127.7.24.212:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 14:53:54.008275 16841 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40205
I20260620 14:53:54.008555 16841 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:54.009094 16841 heartbeater.cc:511] Master 127.7.24.254:40205 requested a full tablet report, sending...
I20260620 14:53:54.010219 16522 ts_manager.cc:194] Registered new tserver with Master: c43d3934fb3e4cd68780dcb8abcdf19a (127.7.24.194:45347)
I20260620 14:53:54.010877 16522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.194:46121
W20260620 14:53:54.123397 16844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:54.123720 16844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:54.123786 16844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:54.127791 16844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:54.127872 16844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:54.127959 16844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:54.132591 16844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:54.133673 16844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:54.135058 16844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:54.144497 16852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:54.144552 16849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:54.144572 16850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:54.144887 16844 server_base.cc:1034] running on GCE node
I20260620 14:53:54.145327 16844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:54.145905 16844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:54.147056 16844 hybrid_clock.cc:648] HybridClock initialized: now 1781967234147027 us; error 44 us; skew 500 ppm
I20260620 14:53:54.148867 16844 webserver.cc:460] Webserver started at http://127.7.24.195:36203/ using document root <none> and password file <none>
I20260620 14:53:54.149389 16844 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:54.149446 16844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:54.149617 16844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:54.151726 16844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "619f9b316b974a2c9ed63552bf33604f"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "7bdbcb15bdfca5d4626ef640d630eeeb"
server_key_iv: "0b2d3aeb6fa47778f1264585ecdb5e22"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.152230 16844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "619f9b316b974a2c9ed63552bf33604f"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "7bdbcb15bdfca5d4626ef640d630eeeb"
server_key_iv: "0b2d3aeb6fa47778f1264585ecdb5e22"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.156252 16844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:53:54.158799 16858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:54.160063 16844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:54.160157 16844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "619f9b316b974a2c9ed63552bf33604f"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "7bdbcb15bdfca5d4626ef640d630eeeb"
server_key_iv: "0b2d3aeb6fa47778f1264585ecdb5e22"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.160297 16844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:54.174670 16844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:54.175370 16844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:54.175630 16844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:54.176224 16844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:54.177211 16844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:54.177258 16844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:54.177325 16844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:54.177357 16844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:54.186307 16844 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:40765
I20260620 14:53:54.186331 16971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:40765 every 8 connection(s)
I20260620 14:53:54.187278 16844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:54.195088 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 16844
I20260620 14:53:54.195200 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:54.195494 7267 external_mini_cluster.cc:1428] Setting key 51f1e13f97d68ffe4844dc6afc1ac4c1
I20260620 14:53:54.196883 16561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:54.189067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:54153 (local address 127.7.24.254:40205)
0620 14:53:54.189210 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:54.189214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:54.190270 (+ 1056us) server_negotiation.cc:407] Connection header received
0620 14:53:54.191707 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:54.191710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:54.191758 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:54.191835 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:54.192798 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:54.193320 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:54.194609 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:54.194893 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:54.195006 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:54.196322 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:54.196328 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:54.196334 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:54.196350 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:54.196373 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:54.196383 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:54.196450 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:54.196643 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 14:53:54.196705 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 14:53:54.197876 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:54.189464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40205 (local address 127.7.24.195:54153)
0620 14:53:54.190123 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:54.190165 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:54.191515 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:54.191993 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:54.191999 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:54.192078 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:54.192664 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:54.192680 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:54.193435 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:54.193438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:54.194317 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:54.194326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:54.195056 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:54.195059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:54.195285 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:54.196010 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:54.196012 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:54.196052 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:54.196150 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:54.196471 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:54.196481 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:54.196487 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 14:53:54.196616 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 14:53:54.196929 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":109,"threads_started":1}
I20260620 14:53:54.199016 16972 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40205
I20260620 14:53:54.199282 16972 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:54.199817 16972 heartbeater.cc:511] Master 127.7.24.254:40205 requested a full tablet report, sending...
I20260620 14:53:54.200912 16522 ts_manager.cc:194] Registered new tserver with Master: 619f9b316b974a2c9ed63552bf33604f (127.7.24.195:40765)
I20260620 14:53:54.201474 16522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.195:54153
I20260620 14:53:54.210170 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 14:53:54.213163 16561 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:54.212344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.7.24.254:40205)
0620 14:53:54.212487 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:54.212490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:54.212677 (+ 187us) server_negotiation.cc:407] Connection header received
0620 14:53:54.212752 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:54.212755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:54.212804 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:54.212869 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:54.213057 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54008: BlockingRecv error: recv got EOF from 127.0.0.1:54008 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 14:53:54.213390 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 16582
I20260620 14:53:54.219027 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 16713
I20260620 14:53:54.224598 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 16844
I20260620 14:53:54.229771 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 16491
2026-06-20T14:53:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (902 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-20T14:53:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:54Z Disabled control of system clock
I20260620 14:53:54.259699 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:46275
--webserver_interface=127.7.24.254
--webserver_port=0
--builtin_ntp_servers=127.7.24.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:46275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 14:53:54.370759 16983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:54.371109 16983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:54.371188 16983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:54.375416 16983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:54.375553 16983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:54.375582 16983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:54.375620 16983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:54.375658 16983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:54.380797 16983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:46275
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:46275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:54.382052 16983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:54.383495 16983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:54.392208 16988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:54.392220 16991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:54.392475 16989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:54.392508 16983 server_base.cc:1034] running on GCE node
I20260620 14:53:54.393168 16983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:54.394308 16983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:54.395524 16983 hybrid_clock.cc:648] HybridClock initialized: now 1781967234395508 us; error 35 us; skew 500 ppm
I20260620 14:53:54.397598 16983 webserver.cc:460] Webserver started at http://127.7.24.254:45973/ using document root <none> and password file <none>
I20260620 14:53:54.398375 16983 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:54.398442 16983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:54.398698 16983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:54.400728 16983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "10b4a8d456c74ce2934921b6cfa6b73b"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "226f6e21f8fdc7982aa3deaca1d9afc1"
server_key_iv: "df33f0c547e4ee41eaca2920aca994fc"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.401232 16983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "10b4a8d456c74ce2934921b6cfa6b73b"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "226f6e21f8fdc7982aa3deaca1d9afc1"
server_key_iv: "df33f0c547e4ee41eaca2920aca994fc"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.404808 16983 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260620 14:53:54.407471 16997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:54.408769 16983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:54.408870 16983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "10b4a8d456c74ce2934921b6cfa6b73b"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "226f6e21f8fdc7982aa3deaca1d9afc1"
server_key_iv: "df33f0c547e4ee41eaca2920aca994fc"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.408957 16983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:54.413415 16983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:54.413983 16983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:54.414125 16983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:54.421463 16983 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:46275
I20260620 14:53:54.421463 17049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:46275 every 8 connection(s)
I20260620 14:53:54.422571 16983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:54.425774 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 16983
I20260620 14:53:54.425870 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:54.425873 17050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:54.426097 7267 external_mini_cluster.cc:1428] Setting key 0845440bd2d7edb20089f4868bf385eb
I20260620 14:53:54.432245 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:54.427448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.7.24.254:46275)
0620 14:53:54.428153 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:54.428173 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:54.428217 (+ 44us) server_negotiation.cc:407] Connection header received
0620 14:53:54.429002 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:54.429026 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:54.429513 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:54.429895 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:54.430696 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:54.430701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:54.430716 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:54.430727 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:54.430791 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:54.430814 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:54.430954 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:54.431027 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 14:53:54.431295 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":109,"threads_started":1}
I20260620 14:53:54.433925 17050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b: Bootstrap starting.
I20260620 14:53:54.436275 17050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:54.437053 17050 log.cc:824] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:54.439219 17050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b: No bootstrap required, opened a new log
I20260620 14:53:54.442986 17050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b4a8d456c74ce2934921b6cfa6b73b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46275 } }
I20260620 14:53:54.443214 17050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:54.443301 17050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10b4a8d456c74ce2934921b6cfa6b73b, State: Initialized, Role: FOLLOWER
I20260620 14:53:54.443743 17050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b4a8d456c74ce2934921b6cfa6b73b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46275 } }
I20260620 14:53:54.443892 17050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:54.443956 17050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:54.444099 17050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:54.445043 17050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b4a8d456c74ce2934921b6cfa6b73b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46275 } }
I20260620 14:53:54.445456 17050 leader_election.cc:302] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10b4a8d456c74ce2934921b6cfa6b73b; no voters:
I20260620 14:53:54.445786 17050 leader_election.cc:288] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:54.445919 17055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:54.446170 17055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [term 1 LEADER]: Becoming Leader. State: Replica: 10b4a8d456c74ce2934921b6cfa6b73b, State: Running, Role: LEADER
I20260620 14:53:54.446503 17055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b4a8d456c74ce2934921b6cfa6b73b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46275 } }
I20260620 14:53:54.447075 17050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:54.448107 17056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10b4a8d456c74ce2934921b6cfa6b73b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b4a8d456c74ce2934921b6cfa6b73b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46275 } } }
I20260620 14:53:54.448236 17056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:54.448504 17057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10b4a8d456c74ce2934921b6cfa6b73b. Latest consensus state: current_term: 1 leader_uuid: "10b4a8d456c74ce2934921b6cfa6b73b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b4a8d456c74ce2934921b6cfa6b73b" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46275 } } }
I20260620 14:53:54.448614 17057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:54.448714 17064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:54.451512 17064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:54.457194 17064 catalog_manager.cc:1347] Generated new cluster ID: ff8bbb55982847a3855ee0fdb585877d
I20260620 14:53:54.457258 17064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:54.478878 17064 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:54.479290 17064 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:54.487075 17064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10b4a8d456c74ce2934921b6cfa6b73b: Generated new TSK 0
I20260620 14:53:54.487926 17064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:53:54.497179 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:46275
--builtin_ntp_servers=127.7.24.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 14:53:54.613539 17074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:54.613811 17074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:54.613850 17074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:54.617945 17074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:54.618012 17074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:54.618069 17074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:54.623116 17074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:46275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:54.624205 17074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:54.625537 17074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:54.635174 17082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:54.635174 17079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:54.635248 17080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:54.635586 17074 server_base.cc:1034] running on GCE node
I20260620 14:53:54.636070 17074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:54.636832 17074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:54.638052 17074 hybrid_clock.cc:648] HybridClock initialized: now 1781967234638032 us; error 37 us; skew 500 ppm
I20260620 14:53:54.640115 17074 webserver.cc:460] Webserver started at http://127.7.24.193:38229/ using document root <none> and password file <none>
I20260620 14:53:54.640727 17074 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:54.640832 17074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:54.641068 17074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:54.643394 17074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "0392365ac1bc40a3a97367ce31f42acc"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "bc78fa3049d608693507bbf687036f4f"
server_key_iv: "0949cd997fadf2d0955daf4974efde11"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.643958 17074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "0392365ac1bc40a3a97367ce31f42acc"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "bc78fa3049d608693507bbf687036f4f"
server_key_iv: "0949cd997fadf2d0955daf4974efde11"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.647728 17074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260620 14:53:54.650588 17088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:54.651983 17074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:53:54.652120 17074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "0392365ac1bc40a3a97367ce31f42acc"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "bc78fa3049d608693507bbf687036f4f"
server_key_iv: "0949cd997fadf2d0955daf4974efde11"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.652242 17074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:54.656858 17074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:54.657454 17074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:54.657637 17074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:54.658157 17074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:54.659092 17074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:54.659163 17074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:54.659233 17074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:54.659284 17074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:54.668742 17074 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:38603
I20260620 14:53:54.668762 17201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:38603 every 8 connection(s)
I20260620 14:53:54.669668 17074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:54.673875 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 17074
I20260620 14:53:54.673972 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:54.674382 7267 external_mini_cluster.cc:1428] Setting key 9652d01a63fc22431f2d91dcad294565
I20260620 14:53:54.676437 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:54.671573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:58865 (local address 127.7.24.254:46275)
0620 14:53:54.671781 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:54.671785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:54.672785 (+ 1000us) server_negotiation.cc:407] Connection header received
0620 14:53:54.674238 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:54.674241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:54.674288 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:54.674395 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:54.675814 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:54.675818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:54.675820 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:54.675860 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:54.675890 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:54.675904 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:54.675995 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:54.676246 (+ 251us) server_negotiation.cc:299] Negotiation successful
0620 14:53:54.676293 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260620 14:53:54.676868 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:46275
--builtin_ntp_servers=127.7.24.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260620 14:53:54.677392 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:54.672000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:46275 (local address 127.7.24.193:58865)
0620 14:53:54.672634 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:54.672674 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:54.674008 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:54.674604 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:54.674613 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:54.674718 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:54.675460 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:54.675462 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:54.675530 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:54.675619 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:54.676007 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:54.676017 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:54.676024 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 14:53:54.676243 (+ 219us) client_negotiation.cc:240] Negotiation successful
0620 14:53:54.676455 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":107,"threads_started":1}
I20260620 14:53:54.679139 17202 heartbeater.cc:347] Connected to a master server at 127.7.24.254:46275
I20260620 14:53:54.679405 17202 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:54.679886 17202 heartbeater.cc:511] Master 127.7.24.254:46275 requested a full tablet report, sending...
I20260620 14:53:54.681443 17014 ts_manager.cc:194] Registered new tserver with Master: 0392365ac1bc40a3a97367ce31f42acc (127.7.24.193:38603)
W20260620 14:53:54.790351 17205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:54.790649 17205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:54.790688 17205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:54.794981 17205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:54.795054 17205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:54.795142 17205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:54.800130 17205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:46275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:54.801339 17205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:54.802876 17205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:54.813336 17213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:54.813336 17210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:54.813450 17211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:54.813788 17205 server_base.cc:1034] running on GCE node
I20260620 14:53:54.814237 17205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:54.815034 17205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:54.816234 17205 hybrid_clock.cc:648] HybridClock initialized: now 1781967234816200 us; error 50 us; skew 500 ppm
I20260620 14:53:54.818313 17205 webserver.cc:460] Webserver started at http://127.7.24.194:33365/ using document root <none> and password file <none>
I20260620 14:53:54.818994 17205 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:54.819092 17205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:54.819342 17205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:54.821762 17205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "4166be37d5384b349047c5ccea00e9ab"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "b42de451e78c7b6537e45ef36c0c467c"
server_key_iv: "23c5f42ef2acdf260f171e90301d899f"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.822427 17205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "4166be37d5384b349047c5ccea00e9ab"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "b42de451e78c7b6537e45ef36c0c467c"
server_key_iv: "23c5f42ef2acdf260f171e90301d899f"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.826557 17205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 14:53:54.829288 17219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:54.830785 17205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260620 14:53:54.830935 17205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "4166be37d5384b349047c5ccea00e9ab"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "b42de451e78c7b6537e45ef36c0c467c"
server_key_iv: "23c5f42ef2acdf260f171e90301d899f"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.831085 17205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:54.835932 17205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:54.836621 17205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:54.836827 17205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:54.837414 17205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:54.838421 17205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:54.838518 17205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:54.838604 17205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:54.838661 17205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:54.848464 17205 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:43633
I20260620 14:53:54.848481 17332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:43633 every 8 connection(s)
I20260620 14:53:54.849465 17205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:54.854771 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 17205
I20260620 14:53:54.854923 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:54.855332 7267 external_mini_cluster.cc:1428] Setting key 9e07ce7bcda6514f1dce74d946266c56
I20260620 14:53:54.856560 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:54.851537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:54637 (local address 127.7.24.254:46275)
0620 14:53:54.851765 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:54.851769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:54.852845 (+ 1076us) server_negotiation.cc:407] Connection header received
0620 14:53:54.854206 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:54.854210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:54.854266 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:54.854354 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:54.855789 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:54.855792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:54.855795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:54.855811 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:54.855839 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:54.855850 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:54.855980 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:54.856328 (+ 348us) server_negotiation.cc:299] Negotiation successful
0620 14:53:54.856373 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260620 14:53:54.857652 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:46275
--builtin_ntp_servers=127.7.24.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260620 14:53:54.857712 17335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:54.851982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:46275 (local address 127.7.24.194:54637)
0620 14:53:54.852685 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:54.852728 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:54.853980 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:54.854605 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:54.854617 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:54.854740 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:54.855421 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:54.855424 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:54.855497 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:54.855592 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:54.855988 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:54.855998 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:54.856004 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 14:53:54.856398 (+ 394us) client_negotiation.cc:240] Negotiation successful
0620 14:53:54.856641 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":109,"threads_started":1}
I20260620 14:53:54.859191 17333 heartbeater.cc:347] Connected to a master server at 127.7.24.254:46275
I20260620 14:53:54.859575 17333 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:54.860123 17333 heartbeater.cc:511] Master 127.7.24.254:46275 requested a full tablet report, sending...
I20260620 14:53:54.861577 17014 ts_manager.cc:194] Registered new tserver with Master: 4166be37d5384b349047c5ccea00e9ab (127.7.24.194:43633)
W20260620 14:53:54.969091 17336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:54.969348 17336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:54.969388 17336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:54.973445 17336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:54.973563 17336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:54.973621 17336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:54.978447 17336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:46275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:54.979631 17336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:54.981021 17336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:54.990552 17341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:54.990636 17342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:54.990639 17344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:54.990764 17336 server_base.cc:1034] running on GCE node
I20260620 14:53:54.991236 17336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:54.991832 17336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:54.993029 17336 hybrid_clock.cc:648] HybridClock initialized: now 1781967234993012 us; error 32 us; skew 500 ppm
I20260620 14:53:54.994989 17336 webserver.cc:460] Webserver started at http://127.7.24.195:43023/ using document root <none> and password file <none>
I20260620 14:53:54.995595 17336 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:54.995685 17336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:54.995913 17336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:54.998042 17336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "b913a3decb2b41ea89e0feb73f6eb00f"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "0c8d7a81c19049c81e7f3222a5d8d154"
server_key_iv: "c7297a3ffb81d621263dddff4bc5a0f0"
server_key_version: "encryptionkey@0"
I20260620 14:53:54.998669 17336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "b913a3decb2b41ea89e0feb73f6eb00f"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "0c8d7a81c19049c81e7f3222a5d8d154"
server_key_iv: "c7297a3ffb81d621263dddff4bc5a0f0"
server_key_version: "encryptionkey@0"
I20260620 14:53:55.002312 17336 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 14:53:55.005039 17350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:55.006356 17336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:55.006539 17336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "b913a3decb2b41ea89e0feb73f6eb00f"
format_stamp: "Formatted at 2026-06-20 14:53:54 on dist-test-slave-sms9"
server_key: "0c8d7a81c19049c81e7f3222a5d8d154"
server_key_iv: "c7297a3ffb81d621263dddff4bc5a0f0"
server_key_version: "encryptionkey@0"
I20260620 14:53:55.006714 17336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:55.011513 17336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:55.012157 17336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:55.012291 17336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:55.012763 17336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:55.013739 17336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:55.013783 17336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:55.013844 17336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:55.013876 17336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:55.022497 17336 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:35061
I20260620 14:53:55.022571 17463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:35061 every 8 connection(s)
I20260620 14:53:55.023447 17336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:55.024915 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 17336
I20260620 14:53:55.025005 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:55.025290 7267 external_mini_cluster.cc:1428] Setting key 26a750abebba63e2345518088ff2fb7e
I20260620 14:53:55.030795 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:55.026300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:59981 (local address 127.7.24.254:46275)
0620 14:53:55.026446 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:55.026451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:55.027795 (+ 1344us) server_negotiation.cc:407] Connection header received
0620 14:53:55.029090 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:55.029094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:55.029136 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:55.029214 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:55.030304 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:55.030308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:55.030311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:53:55.030323 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:55.030345 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:53:55.030361 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:55.030455 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:55.030618 (+ 163us) server_negotiation.cc:299] Negotiation successful
0620 14:53:55.030661 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 14:53:55.031620 17466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:55.026895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:46275 (local address 127.7.24.195:59981)
0620 14:53:55.027647 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:55.027706 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:55.028895 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:55.029372 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:55.029379 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:55.029464 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:55.030051 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:53:55.030053 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:55.030097 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:53:55.030163 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:55.030442 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:55.030451 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:55.030478 (+ 27us) client_negotiation.cc:769] Sending connection context
0620 14:53:55.030658 (+ 180us) client_negotiation.cc:240] Negotiation successful
0620 14:53:55.030888 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":115,"threads_started":1}
I20260620 14:53:55.032721 17464 heartbeater.cc:347] Connected to a master server at 127.7.24.254:46275
I20260620 14:53:55.032969 17464 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:55.033372 17464 heartbeater.cc:511] Master 127.7.24.254:46275 requested a full tablet report, sending...
I20260620 14:53:55.034572 17014 ts_manager.cc:194] Registered new tserver with Master: b913a3decb2b41ea89e0feb73f6eb00f (127.7.24.195:35061)
I20260620 14:53:55.039415 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 14:53:55.042430 17053 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:53:55.041456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45970 (local address 127.7.24.254:46275)
0620 14:53:55.041558 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:55.041561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:55.041807 (+ 246us) server_negotiation.cc:407] Connection header received
0620 14:53:55.041900 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:55.041902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:55.041932 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:55.041993 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:55.042178 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45970: BlockingRecv error: recv got EOF from 127.0.0.1:45970 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 14:53:55.042946 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 17074
I20260620 14:53:55.049225 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 17205
I20260620 14:53:55.054839 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 17336
I20260620 14:53:55.060730 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 16983
2026-06-20T14:53:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (831 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:55 dist-test-slave-sms9 krb5kdc[17474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:55 dist-test-slave-sms9 krb5kdc[17474](info): set up 2 sockets
Jun 20 14:53:55 dist-test-slave-sms9 krb5kdc[17474](info): commencing operation
krb5kdc: starting...
W20260620 14:53:56.107718 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:56 dist-test-slave-sms9 krb5kdc[17474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:53:56.234397 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40015
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:40015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:56.347458 17487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:56.347746 17487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:56.347788 17487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:56.351908 17487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:53:56.351965 17487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:56.351985 17487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:56.352003 17487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:53:56.352018 17487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:53:56.357353 17487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:40015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:56.358479 17487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:56.359506 17487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:56.368067 17493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:56.368067 17492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:56.368156 17487 server_base.cc:1034] running on GCE node
W20260620 14:53:56.368098 17495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:56.368839 17487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:56.369942 17487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:56.371119 17487 hybrid_clock.cc:648] HybridClock initialized: now 1781967236371083 us; error 48 us; skew 500 ppm
Jun 20 14:53:56 dist-test-slave-sms9 krb5kdc[17474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967236, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:56.373782 17487 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:53:56.374822 17487 webserver.cc:460] Webserver started at http://127.7.24.254:44079/ using document root <none> and password file <none>
I20260620 14:53:56.375401 17487 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:56.375514 17487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:56.375756 17487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:56.377816 17487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "8d95de31dc0d4012953529c9122e249c"
format_stamp: "Formatted at 2026-06-20 14:53:56 on dist-test-slave-sms9"
server_key: "8783da3467f0fff2b3e5cf918255d954"
server_key_iv: "5a8b79028f3cb3e866367c8cf1fc4a65"
server_key_version: "encryptionkey@0"
I20260620 14:53:56.378391 17487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "8d95de31dc0d4012953529c9122e249c"
format_stamp: "Formatted at 2026-06-20 14:53:56 on dist-test-slave-sms9"
server_key: "8783da3467f0fff2b3e5cf918255d954"
server_key_iv: "5a8b79028f3cb3e866367c8cf1fc4a65"
server_key_version: "encryptionkey@0"
I20260620 14:53:56.382212 17487 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 14:53:56.384708 17502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:56.385893 17487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:56.386035 17487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "8d95de31dc0d4012953529c9122e249c"
format_stamp: "Formatted at 2026-06-20 14:53:56 on dist-test-slave-sms9"
server_key: "8783da3467f0fff2b3e5cf918255d954"
server_key_iv: "5a8b79028f3cb3e866367c8cf1fc4a65"
server_key_version: "encryptionkey@0"
I20260620 14:53:56.386168 17487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:56.409528 17487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:56.412978 17487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:56.413225 17487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:56.420807 17487 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:40015
I20260620 14:53:56.421128 17554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:40015 every 8 connection(s)
I20260620 14:53:56.421836 17487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:53:56.424815 17555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:56.431279 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 17487
I20260620 14:53:56.431427 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:53:56.431742 7267 external_mini_cluster.cc:1428] Setting key ada9f01e4ddad5d899cfe5bba87ff37e
I20260620 14:53:56.432032 17555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c: Bootstrap starting.
I20260620 14:53:56.434593 17555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:56.435667 17555 log.cc:824] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:56.437866 17555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c: No bootstrap required, opened a new log
Jun 20 14:53:56 dist-test-slave-sms9 krb5kdc[17474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:56.442317 17555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d95de31dc0d4012953529c9122e249c" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40015 } }
I20260620 14:53:56.442629 17555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:56.442684 17555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d95de31dc0d4012953529c9122e249c, State: Initialized, Role: FOLLOWER
I20260620 14:53:56.443158 17555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d95de31dc0d4012953529c9122e249c" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40015 } }
I20260620 14:53:56.443306 17555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:53:56.443392 17555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:53:56.443506 17555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:56.444360 17555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d95de31dc0d4012953529c9122e249c" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40015 } }
I20260620 14:53:56.444705 17555 leader_election.cc:302] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d95de31dc0d4012953529c9122e249c; no voters:
I20260620 14:53:56.445003 17555 leader_election.cc:288] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:53:56.445123 17560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:56.445348 17560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [term 1 LEADER]: Becoming Leader. State: Replica: 8d95de31dc0d4012953529c9122e249c, State: Running, Role: LEADER
I20260620 14:53:56.445667 17560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d95de31dc0d4012953529c9122e249c" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40015 } }
I20260620 14:53:56.446271 17555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:53:56.446835 17560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d95de31dc0d4012953529c9122e249c. Latest consensus state: current_term: 1 leader_uuid: "8d95de31dc0d4012953529c9122e249c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d95de31dc0d4012953529c9122e249c" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40015 } } }
I20260620 14:53:56.446941 17560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:56.447193 17558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:56.433177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.7.24.254:40015)
0620 14:53:56.433608 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:56.433623 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:56.433656 (+ 33us) server_negotiation.cc:407] Connection header received
0620 14:53:56.434250 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:56.434265 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:56.434699 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:56.434988 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:56.435991 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:56.437139 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:56.438298 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:56.438841 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:56.438966 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:56.441664 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:56.441670 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:56.441683 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:56.441710 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:56.443670 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:56.444112 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:56.444119 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:56.444125 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:56.444246 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:56.444530 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:56.444533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:56.444535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:56.445013 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:56.445793 (+ 780us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:56.445936 (+ 143us) server_negotiation.cc:299] Negotiation successful
0620 14:53:56.446217 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":158,"threads_started":1}
I20260620 14:53:56.447544 17561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d95de31dc0d4012953529c9122e249c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d95de31dc0d4012953529c9122e249c" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40015 } } }
I20260620 14:53:56.447635 17561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c [sys.catalog]: This master's current role is: LEADER
I20260620 14:53:56.447889 17565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:53:56.451364 17565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:53:56.456722 17565 catalog_manager.cc:1347] Generated new cluster ID: e6545eac5405425696c66585a7fd0ee4
I20260620 14:53:56.456789 17565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:53:56.469319 17565 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:53:56.470659 17565 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:53:56.481675 17565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c: Generated new TSK 0
I20260620 14:53:56.482450 17565 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:53:56.523238 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40015
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:56.658198 17581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:56.658519 17581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:56.658587 17581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:56.662781 17581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:56.662843 17581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:56.662940 17581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:53:56.668473 17581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:56.669570 17581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:56.670611 17581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:56.680177 17587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:56.680177 17589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:56.680341 17581 server_base.cc:1034] running on GCE node
W20260620 14:53:56.680197 17586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:56.680944 17581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:56.681675 17581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:56.682857 17581 hybrid_clock.cc:648] HybridClock initialized: now 1781967236682841 us; error 28 us; skew 500 ppm
Jun 20 14:53:56 dist-test-slave-sms9 krb5kdc[17474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967236, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:56.685773 17581 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:53:56.686774 17581 webserver.cc:460] Webserver started at http://127.7.24.193:42567/ using document root <none> and password file <none>
I20260620 14:53:56.687387 17581 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:56.687476 17581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:56.687726 17581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:56.689841 17581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "ad72cdbf6b92495fac4cf5595bd09866"
format_stamp: "Formatted at 2026-06-20 14:53:56 on dist-test-slave-sms9"
server_key: "1a9d99e2996a005706aed61225326b10"
server_key_iv: "5745402f28dd204596d1e79666da7c56"
server_key_version: "encryptionkey@0"
I20260620 14:53:56.690433 17581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "ad72cdbf6b92495fac4cf5595bd09866"
format_stamp: "Formatted at 2026-06-20 14:53:56 on dist-test-slave-sms9"
server_key: "1a9d99e2996a005706aed61225326b10"
server_key_iv: "5745402f28dd204596d1e79666da7c56"
server_key_version: "encryptionkey@0"
I20260620 14:53:56.694308 17581 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 14:53:56.696890 17596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:56.698199 17581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:53:56.698364 17581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "ad72cdbf6b92495fac4cf5595bd09866"
format_stamp: "Formatted at 2026-06-20 14:53:56 on dist-test-slave-sms9"
server_key: "1a9d99e2996a005706aed61225326b10"
server_key_iv: "5745402f28dd204596d1e79666da7c56"
server_key_version: "encryptionkey@0"
I20260620 14:53:56.698513 17581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:56.713023 17581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:56.716240 17581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:56.716463 17581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:56.717078 17581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:56.717983 17581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:56.718057 17581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:56.718127 17581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:56.718176 17581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:56.727569 17581 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:37629
I20260620 14:53:56.727622 17709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:37629 every 8 connection(s)
I20260620 14:53:56.728507 17581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:53:56.730988 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 17581
I20260620 14:53:56.731119 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:53:56.731487 7267 external_mini_cluster.cc:1428] Setting key 30b7b3c8b3402a7d2c84fc380f18413a
Jun 20 14:53:56 dist-test-slave-sms9 krb5kdc[17474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967236, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
I20260620 14:53:56.744716 17558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:56.730327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:56525 (local address 127.7.24.254:40015)
0620 14:53:56.730514 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:56.730519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:56.731368 (+ 849us) server_negotiation.cc:407] Connection header received
0620 14:53:56.733086 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:56.733089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:56.733143 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:56.733234 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:56.734488 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:56.735079 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:56.736934 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:56.737240 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:56.737362 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:56.740428 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:56.740434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:56.740441 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:56.740476 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:56.742189 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:56.742931 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:56.742934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:56.742936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:56.743017 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:56.743383 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:56.743386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:56.743387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:56.743616 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:56.743698 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:56.744332 (+ 634us) server_negotiation.cc:299] Negotiation successful
0620 14:53:56.744489 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 14:53:56.745690 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:56.730753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40015 (local address 127.7.24.193:56525)
0620 14:53:56.731196 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:56.731248 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:56.732821 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:56.733393 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:56.733400 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:56.733797 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:56.734274 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:56.734287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:56.735221 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:56.735227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:56.736789 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:56.736805 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:56.737445 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:56.737452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:56.737690 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:56.738325 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:56.738352 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:56.740245 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:56.742299 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:56.742304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:56.742316 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:56.742801 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:56.743146 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:56.743149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:56.743150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:56.743289 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:56.743729 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:56.743734 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:56.743968 (+ 234us) client_negotiation.cc:769] Sending connection context
0620 14:53:56.744219 (+ 251us) client_negotiation.cc:240] Negotiation successful
0620 14:53:56.744557 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":124,"threads_started":1}
I20260620 14:53:56.747242 17710 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40015
I20260620 14:53:56.747514 17710 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:56.748147 17710 heartbeater.cc:511] Master 127.7.24.254:40015 requested a full tablet report, sending...
I20260620 14:53:56.749768 17519 ts_manager.cc:194] Registered new tserver with Master: ad72cdbf6b92495fac4cf5595bd09866 (127.7.24.193:37629)
I20260620 14:53:56.750949 17519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:56525
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:53:56.761790 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40015
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:56.872958 17715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:56.873229 17715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:56.873273 17715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:56.877591 17715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:56.877671 17715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:56.877732 17715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:53:56.883287 17715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:56.884354 17715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:56.885354 17715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:56.894899 17720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:56.894899 17723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:56.894899 17721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:56.895226 17715 server_base.cc:1034] running on GCE node
I20260620 14:53:56.895622 17715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:56.896219 17715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:56.897382 17715 hybrid_clock.cc:648] HybridClock initialized: now 1781967236897353 us; error 43 us; skew 500 ppm
Jun 20 14:53:56 dist-test-slave-sms9 krb5kdc[17474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967236, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:56.900432 17715 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:53:56.901448 17715 webserver.cc:460] Webserver started at http://127.7.24.194:45595/ using document root <none> and password file <none>
I20260620 14:53:56.902024 17715 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:56.902151 17715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:56.902372 17715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:56.904623 17715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "2361a870f8ec47969ecafcd1467b107d"
format_stamp: "Formatted at 2026-06-20 14:53:56 on dist-test-slave-sms9"
server_key: "a768447fb3b4589a62b7046abdf00bc9"
server_key_iv: "26df2a86b4cb21a46f163e4f77dbc5a3"
server_key_version: "encryptionkey@0"
I20260620 14:53:56.905207 17715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "2361a870f8ec47969ecafcd1467b107d"
format_stamp: "Formatted at 2026-06-20 14:53:56 on dist-test-slave-sms9"
server_key: "a768447fb3b4589a62b7046abdf00bc9"
server_key_iv: "26df2a86b4cb21a46f163e4f77dbc5a3"
server_key_version: "encryptionkey@0"
I20260620 14:53:56.909070 17715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:53:56.911525 17730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:56.912721 17715 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:53:56.912866 17715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "2361a870f8ec47969ecafcd1467b107d"
format_stamp: "Formatted at 2026-06-20 14:53:56 on dist-test-slave-sms9"
server_key: "a768447fb3b4589a62b7046abdf00bc9"
server_key_iv: "26df2a86b4cb21a46f163e4f77dbc5a3"
server_key_version: "encryptionkey@0"
I20260620 14:53:56.912981 17715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:56.930897 17715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:56.933995 17715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:56.934216 17715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:56.934880 17715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:56.935729 17715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:56.935802 17715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:56.935873 17715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:56.935925 17715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:56.944988 17715 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:38663
I20260620 14:53:56.945010 17843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:38663 every 8 connection(s)
I20260620 14:53:56.945911 17715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:53:56.948390 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 17715
I20260620 14:53:56.948603 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:53:56.948890 7267 external_mini_cluster.cc:1428] Setting key 8d426e55999e72b0489d2e4097da21e3
Jun 20 14:53:56 dist-test-slave-sms9 krb5kdc[17474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967236, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:53:56.960528 17558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:56.947810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:47095 (local address 127.7.24.254:40015)
0620 14:53:56.947923 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:56.947926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:56.948850 (+ 924us) server_negotiation.cc:407] Connection header received
0620 14:53:56.950568 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:56.950571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:56.950623 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:56.950697 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:56.951857 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:56.952419 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:56.953530 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:56.953902 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:56.954017 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:56.956577 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:56.956583 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:56.956588 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:56.956621 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:56.958422 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:56.958970 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:56.958977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:56.958980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:56.959073 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:56.959410 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:56.959414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:56.959417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:56.959672 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:56.959781 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:56.960250 (+ 469us) server_negotiation.cc:299] Negotiation successful
0620 14:53:56.960394 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 14:53:56.961423 17846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:56.948219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40015 (local address 127.7.24.194:47095)
0620 14:53:56.948685 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:56.948738 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:56.950335 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:56.950860 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:56.950867 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:56.951203 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:56.951668 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:56.951681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:56.952546 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:56.952549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:56.953405 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:56.953412 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:56.954065 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:56.954069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:56.954220 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:56.954873 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:56.954890 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:56.956393 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:56.958574 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:56.958579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:56.958592 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:56.958857 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:56.959173 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:56.959176 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:56.959177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:56.959309 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:56.959785 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:56.959790 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:56.960027 (+ 237us) client_negotiation.cc:769] Sending connection context
0620 14:53:56.960197 (+ 170us) client_negotiation.cc:240] Negotiation successful
0620 14:53:56.960513 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":129,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
I20260620 14:53:56.962744 17844 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40015
I20260620 14:53:56.962978 17844 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:56.963478 17844 heartbeater.cc:511] Master 127.7.24.254:40015 requested a full tablet report, sending...
I20260620 14:53:56.964537 17519 ts_manager.cc:194] Registered new tserver with Master: 2361a870f8ec47969ecafcd1467b107d (127.7.24.194:38663)
I20260620 14:53:56.965425 17519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:47095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:53:56.978844 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40015
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:53:57.091223 17849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:53:57.091472 17849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:53:57.091567 17849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:53:57.095783 17849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:53:57.095842 17849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:53:57.095933 17849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:53:57.101276 17849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:53:57.102429 17849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:53:57.103539 17849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:53:57.113514 17854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:57.113514 17857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:53:57.113579 17855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:53:57.113629 17849 server_base.cc:1034] running on GCE node
I20260620 14:53:57.114300 17849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:53:57.115005 17849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:53:57.116184 17849 hybrid_clock.cc:648] HybridClock initialized: now 1781967237116171 us; error 26 us; skew 500 ppm
Jun 20 14:53:57 dist-test-slave-sms9 krb5kdc[17474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967237, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:53:57.123807 17849 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:53:57.124804 17849 webserver.cc:460] Webserver started at http://127.7.24.195:33555/ using document root <none> and password file <none>
I20260620 14:53:57.125388 17849 fs_manager.cc:359] Metadata directory not provided
I20260620 14:53:57.125478 17849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:53:57.125715 17849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:53:57.127830 17849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "5c24ed14032c4b4eabf1c265cc26c84a"
format_stamp: "Formatted at 2026-06-20 14:53:57 on dist-test-slave-sms9"
server_key: "f3e344ddcb37e22b4b107a2db6d865b7"
server_key_iv: "e4579ec4ca91ea4500431361e45a9fb4"
server_key_version: "encryptionkey@0"
I20260620 14:53:57.128405 17849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "5c24ed14032c4b4eabf1c265cc26c84a"
format_stamp: "Formatted at 2026-06-20 14:53:57 on dist-test-slave-sms9"
server_key: "f3e344ddcb37e22b4b107a2db6d865b7"
server_key_iv: "e4579ec4ca91ea4500431361e45a9fb4"
server_key_version: "encryptionkey@0"
I20260620 14:53:57.132123 17849 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260620 14:53:57.134534 17864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:57.135758 17849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 14:53:57.135896 17849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "5c24ed14032c4b4eabf1c265cc26c84a"
format_stamp: "Formatted at 2026-06-20 14:53:57 on dist-test-slave-sms9"
server_key: "f3e344ddcb37e22b4b107a2db6d865b7"
server_key_iv: "e4579ec4ca91ea4500431361e45a9fb4"
server_key_version: "encryptionkey@0"
I20260620 14:53:57.136009 17849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:53:57.156592 17849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:53:57.160153 17849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:53:57.160374 17849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:53:57.160977 17849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:53:57.161819 17849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:53:57.161895 17849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:57.161965 17849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:53:57.162007 17849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:53:57.171327 17849 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:42795
I20260620 14:53:57.171353 17977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:42795 every 8 connection(s)
I20260620 14:53:57.172235 17849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:53:57.175778 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 17849
I20260620 14:53:57.175947 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:53:57.176290 7267 external_mini_cluster.cc:1428] Setting key d9c96ef7e11dc801613a50079cf24f9d
Jun 20 14:53:57 dist-test-slave-sms9 krb5kdc[17474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967237, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:53:57.186352 17558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.174069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:36719 (local address 127.7.24.254:40015)
0620 14:53:57.174242 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:57.174245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:57.175067 (+ 822us) server_negotiation.cc:407] Connection header received
0620 14:53:57.176651 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:57.176654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:57.176706 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:57.176814 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:57.178158 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.178834 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.179956 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.180186 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.180300 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.182834 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:57.182838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:57.182841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:57.182873 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:57.184354 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:57.184859 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:57.184864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:57.184865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:57.184947 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:57.185303 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:57.185308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:57.185310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:57.185488 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:57.185598 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:57.186074 (+ 476us) server_negotiation.cc:299] Negotiation successful
0620 14:53:57.186191 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 14:53:57.187237 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.174438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40015 (local address 127.7.24.195:36719)
0620 14:53:57.174906 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:57.174951 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:57.176420 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:57.176977 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:57.176984 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:57.177443 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:53:57.178007 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.178020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.178972 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.178975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.179824 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.179833 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.180338 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.180341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.180511 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.181138 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:53:57.181154 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:53:57.182676 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:53:57.184478 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:57.184484 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:57.184500 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:57.184754 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:57.185055 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:53:57.185058 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:53:57.185060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:53:57.185207 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:53:57.185596 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:53:57.185603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:53:57.185843 (+ 240us) client_negotiation.cc:769] Sending connection context
0620 14:53:57.186042 (+ 199us) client_negotiation.cc:240] Negotiation successful
0620 14:53:57.186352 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":104,"threads_started":1}
I20260620 14:53:57.188306 17978 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40015
I20260620 14:53:57.188550 17978 heartbeater.cc:464] Registering TS with master...
I20260620 14:53:57.189213 17978 heartbeater.cc:511] Master 127.7.24.254:40015 requested a full tablet report, sending...
I20260620 14:53:57.190276 17518 ts_manager.cc:194] Registered new tserver with Master: 5c24ed14032c4b4eabf1c265cc26c84a (127.7.24.195:42795)
I20260620 14:53:57.191009 17518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:36719
I20260620 14:53:57.202368 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:53:57.212817 17558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.204484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58714 (local address 127.7.24.254:40015)
0620 14:53:57.204665 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:57.204668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:57.204764 (+ 96us) server_negotiation.cc:407] Connection header received
0620 14:53:57.204834 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:57.204836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:57.204886 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:57.204969 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:53:57.205788 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.206330 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.207396 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.207679 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.207779 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.209229 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:53:57.209232 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:53:57.209235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:53:57.209262 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:53:57.211050 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:57.211502 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:57.211507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:57.211508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:57.211596 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:53:57.211874 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:53:57.211877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:53:57.211879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:53:57.212131 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:53:57.212278 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:57.212506 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 14:53:57.212625 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260620 14:53:57.216398 17518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:53:57.219969 17518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:53:57.234648 17993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.228890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38663 (local address 127.0.0.1:54488)
0620 14:53:57.229916 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:57.229931 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:57.230129 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:57.230834 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:57.230839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:57.230857 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:57.230961 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.230966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.232044 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.232050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.233196 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.233208 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.234050 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.234054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.234167 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.234205 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 14:53:57.234322 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 14:53:57.234444 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"mutex_wait_us":54,"thread_start_us":104,"threads_started":1}
I20260620 14:53:57.234700 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.229175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54488 (local address 127.7.24.194:38663)
0620 14:53:57.229757 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:57.229777 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:57.230012 (+ 235us) server_negotiation.cc:407] Connection header received
0620 14:53:57.230322 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:57.230330 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:57.230565 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:57.230669 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:57.231077 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.231917 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.233362 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.233918 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.234041 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.234207 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:57.234402 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 14:53:57.234542 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":112,"threads_started":1}
I20260620 14:53:57.235662 17990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.228890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:37629 (local address 127.0.0.1:47930)
0620 14:53:57.229866 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:57.229922 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:57.230129 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:57.230939 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:57.230942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:57.230959 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:57.231094 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.231102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.232727 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.232731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.234113 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.234124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.235267 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.235271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.235355 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.235377 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 14:53:57.235437 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 14:53:57.235520 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":5,"spinlock_wait_cycles":3456,"thread_start_us":108,"threads_started":1}
I20260620 14:53:57.235705 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.229024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47930 (local address 127.7.24.193:37629)
0620 14:53:57.229409 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:57.229424 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:57.230017 (+ 593us) server_negotiation.cc:407] Connection header received
0620 14:53:57.230339 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:57.230346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:57.230594 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:57.230734 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:57.231236 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.232548 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.234281 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.235136 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.235272 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.235410 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:57.235484 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 14:53:57.235572 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":91,"threads_started":1}
I20260620 14:53:57.236078 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.229686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:42795 (local address 127.0.0.1:49552)
0620 14:53:57.230280 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:57.230293 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:57.230360 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:57.231110 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:57.231114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:57.231136 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:57.231219 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.231224 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.232728 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.232731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.234260 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.234271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.235662 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.235666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.235770 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.235793 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 14:53:57.235870 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 14:53:57.235939 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":46,"thread_start_us":92,"threads_started":1}
I20260620 14:53:57.236559 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.229887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.7.24.195:42795)
0620 14:53:57.230647 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:57.230663 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:57.230689 (+ 26us) server_negotiation.cc:407] Connection header received
0620 14:53:57.230764 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:57.230771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:57.230961 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:57.231110 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:57.231319 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.232589 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.234404 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.235299 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.235414 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.235591 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:57.236324 (+ 733us) server_negotiation.cc:299] Negotiation successful
0620 14:53:57.236423 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":165,"threads_started":1}
I20260620 14:53:57.238765 17912 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb65a98c0484dc88870eff01e2ef789 (DEFAULT_TABLE table=test-table [id=3cfd69ed906445b3a1ef3be45503aac2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:57.238729 17644 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb65a98c0484dc88870eff01e2ef789 (DEFAULT_TABLE table=test-table [id=3cfd69ed906445b3a1ef3be45503aac2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:57.238729 17778 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb65a98c0484dc88870eff01e2ef789 (DEFAULT_TABLE table=test-table [id=3cfd69ed906445b3a1ef3be45503aac2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:53:57.239874 17644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb65a98c0484dc88870eff01e2ef789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:57.239874 17778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb65a98c0484dc88870eff01e2ef789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:57.239889 17912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb65a98c0484dc88870eff01e2ef789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:53:57.246215 17995 tablet_bootstrap.cc:492] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a: Bootstrap starting.
I20260620 14:53:57.248093 17996 tablet_bootstrap.cc:492] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866: Bootstrap starting.
I20260620 14:53:57.248373 17997 tablet_bootstrap.cc:492] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d: Bootstrap starting.
I20260620 14:53:57.249392 17995 tablet_bootstrap.cc:654] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:57.250136 17995 log.cc:824] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:57.250435 17996 tablet_bootstrap.cc:654] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:57.250957 17997 tablet_bootstrap.cc:654] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d: Neither blocks nor log segments found. Creating new log.
I20260620 14:53:57.251421 17996 log.cc:824] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:57.251788 17997 log.cc:824] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d: Log is configured to *not* fsync() on all Append() calls
I20260620 14:53:57.252790 17995 tablet_bootstrap.cc:492] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a: No bootstrap required, opened a new log
I20260620 14:53:57.253010 17995 ts_tablet_manager.cc:1397] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260620 14:53:57.253346 17996 tablet_bootstrap.cc:492] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866: No bootstrap required, opened a new log
I20260620 14:53:57.253553 17996 ts_tablet_manager.cc:1397] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260620 14:53:57.253778 17997 tablet_bootstrap.cc:492] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d: No bootstrap required, opened a new log
I20260620 14:53:57.253957 17997 ts_tablet_manager.cc:1397] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:53:57.256189 17995 raft_consensus.cc:348] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad72cdbf6b92495fac4cf5595bd09866" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37629 } } peers { permanent_uuid: "2361a870f8ec47969ecafcd1467b107d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38663 } } peers { permanent_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 42795 } }
I20260620 14:53:57.256460 17995 raft_consensus.cc:374] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:57.256513 17995 raft_consensus.cc:729] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c24ed14032c4b4eabf1c265cc26c84a, State: Initialized, Role: FOLLOWER
I20260620 14:53:57.256950 17995 consensus_queue.cc:260] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad72cdbf6b92495fac4cf5595bd09866" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37629 } } peers { permanent_uuid: "2361a870f8ec47969ecafcd1467b107d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38663 } } peers { permanent_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 42795 } }
I20260620 14:53:57.257732 17978 heartbeater.cc:503] Master 127.7.24.254:40015 was elected leader, sending a full tablet report...
I20260620 14:53:57.258271 17997 raft_consensus.cc:348] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad72cdbf6b92495fac4cf5595bd09866" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37629 } } peers { permanent_uuid: "2361a870f8ec47969ecafcd1467b107d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38663 } } peers { permanent_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 42795 } }
I20260620 14:53:57.258272 17996 raft_consensus.cc:348] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad72cdbf6b92495fac4cf5595bd09866" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37629 } } peers { permanent_uuid: "2361a870f8ec47969ecafcd1467b107d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38663 } } peers { permanent_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 42795 } }
I20260620 14:53:57.258558 17997 raft_consensus.cc:374] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:57.258587 17996 raft_consensus.cc:374] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:53:57.258594 17995 ts_tablet_manager.cc:1428] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260620 14:53:57.258630 17997 raft_consensus.cc:729] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2361a870f8ec47969ecafcd1467b107d, State: Initialized, Role: FOLLOWER
I20260620 14:53:57.258646 17996 raft_consensus.cc:729] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad72cdbf6b92495fac4cf5595bd09866, State: Initialized, Role: FOLLOWER
I20260620 14:53:57.259094 17997 consensus_queue.cc:260] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad72cdbf6b92495fac4cf5595bd09866" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37629 } } peers { permanent_uuid: "2361a870f8ec47969ecafcd1467b107d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38663 } } peers { permanent_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 42795 } }
I20260620 14:53:57.259094 17996 consensus_queue.cc:260] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad72cdbf6b92495fac4cf5595bd09866" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37629 } } peers { permanent_uuid: "2361a870f8ec47969ecafcd1467b107d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38663 } } peers { permanent_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 42795 } }
I20260620 14:53:57.259778 17710 heartbeater.cc:503] Master 127.7.24.254:40015 was elected leader, sending a full tablet report...
I20260620 14:53:57.260090 17996 ts_tablet_manager.cc:1428] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260620 14:53:57.260094 17997 ts_tablet_manager.cc:1428] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260620 14:53:57.260425 17844 heartbeater.cc:503] Master 127.7.24.254:40015 was elected leader, sending a full tablet report...
I20260620 14:53:57.280225 18001 raft_consensus.cc:482] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:53:57.280449 18001 raft_consensus.cc:504] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad72cdbf6b92495fac4cf5595bd09866" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37629 } } peers { permanent_uuid: "2361a870f8ec47969ecafcd1467b107d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38663 } } peers { permanent_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 42795 } }
I20260620 14:53:57.281988 18001 leader_election.cc:288] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad72cdbf6b92495fac4cf5595bd09866 (127.7.24.193:37629), 2361a870f8ec47969ecafcd1467b107d (127.7.24.194:38663)
I20260620 14:53:57.287405 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.282443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:52499 (local address 127.7.24.193:37629)
0620 14:53:57.282586 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:57.282592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:57.282607 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:53:57.282662 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:57.282665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:57.282714 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:57.282781 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:57.283154 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.283973 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.285983 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.286628 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.286762 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.286814 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:57.287203 (+ 389us) server_negotiation.cc:299] Negotiation successful
0620 14:53:57.287274 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 14:53:57.287405 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.282376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:34087 (local address 127.7.24.194:38663)
0620 14:53:57.282524 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:57.282529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:57.282873 (+ 344us) server_negotiation.cc:407] Connection header received
0620 14:53:57.282950 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:57.282952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:57.282991 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:57.283053 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:53:57.283314 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.284036 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.285983 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.286671 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.286773 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.286814 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:57.287203 (+ 389us) server_negotiation.cc:299] Negotiation successful
0620 14:53:57.287274 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:53:57.287447 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.282224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:37629 (local address 127.7.24.195:52499)
0620 14:53:57.282349 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:57.282366 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:57.282490 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:57.282831 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:57.282842 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:57.282867 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:57.283004 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.283010 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.284127 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.284131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.285842 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.285854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.286777 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.286781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.287028 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.287096 (+ 68us) client_negotiation.cc:769] Sending connection context
0620 14:53:57.287179 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 14:53:57.287286 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":315,"spinlock_wait_cycles":384}
I20260620 14:53:57.287460 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.282227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:38663 (local address 127.7.24.195:34087)
0620 14:53:57.282743 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:57.282753 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 14:53:57.282877 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:53:57.283090 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:53:57.283094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:53:57.283116 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:53:57.283211 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.283215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.284131 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.284134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.285842 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:53:57.285854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.286777 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.286781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:53:57.287047 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.287092 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 14:53:57.287177 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 14:53:57.287285 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":392,"spinlock_wait_cycles":6400,"thread_start_us":170,"threads_started":1}
I20260620 14:53:57.288178 17664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb65a98c0484dc88870eff01e2ef789" candidate_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad72cdbf6b92495fac4cf5595bd09866" is_pre_election: true
I20260620 14:53:57.288220 17798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb65a98c0484dc88870eff01e2ef789" candidate_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2361a870f8ec47969ecafcd1467b107d" is_pre_election: true
I20260620 14:53:57.288507 17664 raft_consensus.cc:2454] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c24ed14032c4b4eabf1c265cc26c84a in term 0.
I20260620 14:53:57.288519 17798 raft_consensus.cc:2454] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c24ed14032c4b4eabf1c265cc26c84a in term 0.
I20260620 14:53:57.289037 17868 leader_election.cc:302] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2361a870f8ec47969ecafcd1467b107d, 5c24ed14032c4b4eabf1c265cc26c84a, ad72cdbf6b92495fac4cf5595bd09866; no voters:
I20260620 14:53:57.289347 18001 raft_consensus.cc:2781] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:53:57.289424 18001 raft_consensus.cc:482] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:53:57.289455 18001 raft_consensus.cc:3037] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:57.290742 18001 raft_consensus.cc:504] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad72cdbf6b92495fac4cf5595bd09866" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37629 } } peers { permanent_uuid: "2361a870f8ec47969ecafcd1467b107d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38663 } } peers { permanent_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 42795 } }
I20260620 14:53:57.291129 18001 leader_election.cc:288] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [CANDIDATE]: Term 1 election: Requested vote from peers ad72cdbf6b92495fac4cf5595bd09866 (127.7.24.193:37629), 2361a870f8ec47969ecafcd1467b107d (127.7.24.194:38663)
I20260620 14:53:57.291471 17664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb65a98c0484dc88870eff01e2ef789" candidate_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad72cdbf6b92495fac4cf5595bd09866"
I20260620 14:53:57.291500 17798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb65a98c0484dc88870eff01e2ef789" candidate_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2361a870f8ec47969ecafcd1467b107d"
I20260620 14:53:57.291610 17798 raft_consensus.cc:3037] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:57.291610 17664 raft_consensus.cc:3037] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:53:57.292639 17798 raft_consensus.cc:2454] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c24ed14032c4b4eabf1c265cc26c84a in term 1.
I20260620 14:53:57.292645 17664 raft_consensus.cc:2454] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c24ed14032c4b4eabf1c265cc26c84a in term 1.
I20260620 14:53:57.292936 17865 leader_election.cc:302] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2361a870f8ec47969ecafcd1467b107d, 5c24ed14032c4b4eabf1c265cc26c84a; no voters:
I20260620 14:53:57.293133 18001 raft_consensus.cc:2781] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:53:57.293370 18001 raft_consensus.cc:686] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 1 LEADER]: Becoming Leader. State: Replica: 5c24ed14032c4b4eabf1c265cc26c84a, State: Running, Role: LEADER
I20260620 14:53:57.293721 18001 consensus_queue.cc:237] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad72cdbf6b92495fac4cf5595bd09866" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37629 } } peers { permanent_uuid: "2361a870f8ec47969ecafcd1467b107d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38663 } } peers { permanent_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 42795 } }
I20260620 14:53:57.296885 17519 catalog_manager.cc:5495] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c24ed14032c4b4eabf1c265cc26c84a (127.7.24.195). New cstate: current_term: 1 leader_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad72cdbf6b92495fac4cf5595bd09866" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2361a870f8ec47969ecafcd1467b107d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c24ed14032c4b4eabf1c265cc26c84a" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 42795 } health_report { overall_health: HEALTHY } } }
I20260620 14:53:57.314630 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.311168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49566 (local address 127.7.24.195:42795)
0620 14:53:57.311285 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 14:53:57.311290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:53:57.311304 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:53:57.311450 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:53:57.311453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:53:57.311509 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:53:57.311600 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:53:57.311867 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.312364 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.313192 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:53:57.313394 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:53:57.313469 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:53:57.313738 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:53:57.314159 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:53:57.314248 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 14:53:57.314361 (+ 113us) server_negotiation.cc:299] Negotiation successful
0620 14:53:57.314411 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 14:53:57.320586 17798 raft_consensus.cc:1261] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d [term 1 FOLLOWER]: Refusing update from remote peer 5c24ed14032c4b4eabf1c265cc26c84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:57.320586 17664 raft_consensus.cc:1261] T 8fb65a98c0484dc88870eff01e2ef789 P ad72cdbf6b92495fac4cf5595bd09866 [term 1 FOLLOWER]: Refusing update from remote peer 5c24ed14032c4b4eabf1c265cc26c84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:53:57.321341 18005 consensus_queue.cc:1035] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2361a870f8ec47969ecafcd1467b107d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 38663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:57.321560 18001 consensus_queue.cc:1035] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad72cdbf6b92495fac4cf5595bd09866" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:53:57.329473 18010 mvcc.cc:205] Tried to move back new op lower bound from 7298937804058095616 to 7298937803957891072. Current Snapshot: MvccSnapshot[applied={T|T < 7298937804058095616}]
I20260620 14:53:57.339166 17518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:57.339397 17518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:53:57.342083 17518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d95de31dc0d4012953529c9122e249c: Sending DeleteTablet for 3 replicas of tablet 8fb65a98c0484dc88870eff01e2ef789
I20260620 14:53:57.342963 17912 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb65a98c0484dc88870eff01e2ef789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:57 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:49552
I20260620 14:53:57.342967 17778 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb65a98c0484dc88870eff01e2ef789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:57 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54488
I20260620 14:53:57.343077 17644 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb65a98c0484dc88870eff01e2ef789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:53:57 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:47930
I20260620 14:53:57.343528 18017 tablet_replica.cc:330] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a: stopping tablet replica
I20260620 14:53:57.343631 18018 tablet_replica.cc:330] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d: stopping tablet replica
I20260620 14:53:57.343855 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 17581
I20260620 14:53:57.343876 18018 raft_consensus.cc:2229] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:53:57.343868 18017 raft_consensus.cc:2229] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:53:57.344818 18017 raft_consensus.cc:2258] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:57.344978 18018 raft_consensus.cc:2258] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:53:57.346423 18017 ts_tablet_manager.cc:1905] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:57.346423 18018 ts_tablet_manager.cc:1905] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:53:57.349238 18018 ts_tablet_manager.cc:1918] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:57.349275 18017 ts_tablet_manager.cc:1918] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:53:57.349349 18018 log.cc:1196] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/8fb65a98c0484dc88870eff01e2ef789
I20260620 14:53:57.349354 18017 log.cc:1196] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/8fb65a98c0484dc88870eff01e2ef789
I20260620 14:53:57.349702 18018 ts_tablet_manager.cc:1939] T 8fb65a98c0484dc88870eff01e2ef789 P 2361a870f8ec47969ecafcd1467b107d: Deleting consensus metadata
I20260620 14:53:57.349702 18017 ts_tablet_manager.cc:1939] T 8fb65a98c0484dc88870eff01e2ef789 P 5c24ed14032c4b4eabf1c265cc26c84a: Deleting consensus metadata
I20260620 14:53:57.350970 17503 catalog_manager.cc:4841] TS 2361a870f8ec47969ecafcd1467b107d (127.7.24.194:38663): tablet 8fb65a98c0484dc88870eff01e2ef789 (table test-table [id=3cfd69ed906445b3a1ef3be45503aac2]) successfully deleted
I20260620 14:53:57.350978 17506 catalog_manager.cc:4841] TS 5c24ed14032c4b4eabf1c265cc26c84a (127.7.24.195:42795): tablet 8fb65a98c0484dc88870eff01e2ef789 (table test-table [id=3cfd69ed906445b3a1ef3be45503aac2]) successfully deleted
W20260620 14:53:57.355113 17506 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:37629 (error 108)
I20260620 14:53:57.355253 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 17715
I20260620 14:53:57.356315 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:53:57.356016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:37629 (local address 127.0.0.1:47938)
0620 14:53:57.356159 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 14:53:57.356231 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:37629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260620 14:53:57.356458 17506 catalog_manager.cc:4584] TS ad72cdbf6b92495fac4cf5595bd09866 (127.7.24.193:37629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8fb65a98c0484dc88870eff01e2ef789: Network error: Client connection negotiation failed: client connection to 127.7.24.193:37629: connect: Connection refused (error 111)
I20260620 14:53:57.360962 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 17849
I20260620 14:53:57.365957 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 17487
2026-06-20T14:53:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2304 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:53:57 dist-test-slave-sms9 krb5kdc[18023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:53:57 dist-test-slave-sms9 krb5kdc[18023](info): set up 2 sockets
Jun 20 14:53:57 dist-test-slave-sms9 krb5kdc[18023](info): commencing operation
krb5kdc: starting...
W20260620 14:53:58.423650 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:53:58 dist-test-slave-sms9 krb5kdc[18023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:53:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:53:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.24.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.24.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/rangeradmin_127.7.24.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/rangeradmin_127.7.24.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.24.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.24.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/rangerlookup_127.7.24.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/rangerlookup_127.7.24.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.24.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.24.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/HTTP_127.7.24.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/HTTP_127.7.24.212@KRBTEST.COM.keytab.
I20260620 14:53:58.596601 7267 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskNGdDYa/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/postgres -l logfile start
2026-06-20 14:54:01.196 UTC [18053] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 14:54:01.196 UTC [18053] LOG: listening on IPv4 address "127.7.24.212", port 39737
2026-06-20 14:54:01.200 UTC [18053] LOG: listening on Unix socket "/tmp/.s.PGSQL.39737"
2026-06-20 14:54:01.227 UTC [18056] LOG: database system was shut down at 2026-06-20 14:53:59 UTC
2026-06-20 14:54:01.232 UTC [18053] LOG: database system is ready to accept connections
I20260620 14:54:02.187438 7267 mini_postgres.cc:96] Postgres bound to 39737
2026-06-20 14:54:02.194 UTC [18064] FATAL: database "slave" does not exist
127.7.24.212:39737 - accepting connections
I20260620 14:54:02.195768 7267 mini_ranger.cc:161] Starting Ranger...
I20260620 14:54:02.214207 7267 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 14:54:02.726080 7267 mini_ranger.cc:87] Created ranger Postgres database
I20260620 14:54:02.726198 7267 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-admin
2026-06-20 14:54:03,036 [I] DB FLAVOR :POSTGRES
2026-06-20 14:54:03,036 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 14:54:03,036 [I] Checking connection..
2026-06-20 14:54:03,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 14:54:03,307 [I] Checking connection passed.
2026-06-20 14:54:03,307 [I] --------- Verifying version history table ---------
2026-06-20 14:54:03,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 14:54:03,559 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 14:54:03,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 14:54:03,806 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 14:54:03,806 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 14:54:03,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 14:54:04.058 UTC [18128] WARNING: there is no transaction in progress
2026-06-20 14:54:04,064 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 14:54:04,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 14:54:04,303 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 14:54:04,303 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 14:54:04,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 14:54:04,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 14:54:04,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sms9.c.gcp-upstream.internal','N') ;"
2026-06-20 14:54:05,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 14:54:05,334 [I] Table x_portal_user does not exist in database ranger
2026-06-20 14:54:05,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 14:54:05,582 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 14:54:05,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 14:54:05,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 14:54:06,071 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 14:54:06,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 14:54:07.071 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.088 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.105 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.122 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.139 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.158 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.215 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.223 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.239 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.250 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.261 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.274 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.285 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.291 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.299 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07.306 UTC [18236] WARNING: there is no transaction in progress
2026-06-20 14:54:07,981 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 14:54:07,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 14:54:08,204 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 14:54:08,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 14:54:08,487 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 14:54:08,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 14:54:08,738 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 14:54:08,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 14:54:08,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 14:54:09,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sms9.c.gcp-upstream.internal';"
2026-06-20 14:54:09,463 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 14:54:09,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 14:54:09,697 [I] DB_PATCHES have already been applied
I20260620 14:54:09.702976 7267 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-admin:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskNGdDYa/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 14:54:09.703051 7267 mini_ranger.cc:193] Using host: 127.7.24.212
I20260620 14:54:09.706734 7267 mini_ranger.cc:227] Ranger admin URL: http://127.7.24.212:60347
Jun 20, 2026 2:54:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 2:54:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 2:54:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 2:54:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 2:54:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 2:54:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 2:54:10 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 14:54:10.822291 7267 mini_ranger.cc:160] Time spent starting Ranger: real 8.627s user 0.006s sys 0.006s
Jun 20, 2026 2:54:31 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 2:54:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 20, 2026 2:54:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 2:54:31 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 2:54:32 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 14:54:33.449908 7267 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.24.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.24.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/rangerkms_127.7.24.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/rangerkms_127.7.24.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.24.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.24.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.24.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/HTTP_127.7.24.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.24.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/HTTP_127.7.24.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 14:54:33 dist-test-slave-sms9 krb5kdc[18023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967273, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 14:54:33.540531 7267 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 14:54:33.551474 7267 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 14:54:34.433285 7267 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 14:54:34.433387 7267 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-kms
I20260620 14:54:34.433496 7267 mini_ranger_kms.cc:227] Using postgres at 127.7.24.212:39737
2026-06-20 14:54:34,608 [I] DB FLAVOR :POSTGRES
2026-06-20 14:54:34,608 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 14:54:34,608 [I] Checking connection
2026-06-20 14:54:34,608 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 14:54:34,873 [I] connection success
2026-06-20 14:54:34,873 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 14:54:34,873 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 14:54:34,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 14:54:35,119 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 14:54:35,120 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 14:54:35,120 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 14:54:35,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:39737/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 14:54:35,398 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 14:54:35.879343 7267 mini_ranger_kms.cc:310] Created kms service
I20260620 14:54:36.077310 7267 mini_ranger_kms.cc:326] Created kudu user
I20260620 14:54:36.120455 7267 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 14:54:36 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967273, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:54:36.513082 7267 mini_ranger_kms.cc:384] Added ranger policy
I20260620 14:54:36.513232 7267 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-kms:/tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 14:54:36.513252 7267 mini_ranger_kms.cc:240] Using host: 127.7.24.212
I20260620 14:54:36.517349 7267 mini_ranger_kms.cc:276] Ranger KMS PID: 18416
I20260620 14:54:36.517505 7267 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.24.212:41869
Jun 20, 2026 2:54:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 2:54:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 2:54:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 2:54:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 2:54:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 2:54:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 20 14:54:38 dist-test-slave-sms9 krb5kdc[18023](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781967278, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.24.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:54:38.794817 7267 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.254s user 0.002s sys 0.155s
I20260620 14:54:38.795004 7267 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 14:54:38.795136 7267 mini_ranger_kms.cc:401] 127.7.24.212:41869/kms/v1/keys
Jun 20 14:54:39 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967278, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.24.212@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
Jun 20 14:54:39 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967278, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.24.212@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
Jun 20 14:54:41 dist-test-slave-sms9 krb5kdc[18023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:54:41.737686 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:42343
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:42343
--ranger_config_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:41869/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:54:41.853209 18467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:54:41.853477 18467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:54:41.853518 18467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:54:41.857498 18467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:54:41.857560 18467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:54:41.857579 18467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:54:41.857595 18467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:54:41.857610 18467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:54:41.863231 18467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.24.212:41869/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:42343
--ranger_config_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:42343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:54:41.864315 18467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:54:41.865300 18467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:54:41.874264 18467 server_base.cc:1034] running on GCE node
W20260620 14:54:41.874244 18473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:54:41.874226 18472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:54:41.874226 18475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:54:41.875102 18467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:54:41.876242 18467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:54:41.877431 18467 hybrid_clock.cc:648] HybridClock initialized: now 1781967281877421 us; error 35 us; skew 500 ppm
Jun 20 14:54:41 dist-test-slave-sms9 krb5kdc[18023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967281, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:54:41.880157 18467 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:54:41.880985 18467 webserver.cc:460] Webserver started at http://127.7.24.254:42281/ using document root <none> and password file <none>
I20260620 14:54:41.881481 18467 fs_manager.cc:359] Metadata directory not provided
I20260620 14:54:41.881533 18467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:54:41.881704 18467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 14:54:41 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967281, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:54:41.945809 18467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "7ab1bc7be0044ebd93e84450cbef62fc"
format_stamp: "Formatted at 2026-06-20 14:54:41 on dist-test-slave-sms9"
server_key: "1f12b2ca401240d9515777b21d674103"
server_key_iv: "f8d2e083d9306af9955b7f8cb21e8223"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:41.946439 18467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "7ab1bc7be0044ebd93e84450cbef62fc"
format_stamp: "Formatted at 2026-06-20 14:54:41 on dist-test-slave-sms9"
server_key: "1f12b2ca401240d9515777b21d674103"
server_key_iv: "f8d2e083d9306af9955b7f8cb21e8223"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:41.950390 18467 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260620 14:54:41.968005 18482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:54:41.969419 18467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 14:54:41.969571 18467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "7ab1bc7be0044ebd93e84450cbef62fc"
format_stamp: "Formatted at 2026-06-20 14:54:41 on dist-test-slave-sms9"
server_key: "1f12b2ca401240d9515777b21d674103"
server_key_iv: "f8d2e083d9306af9955b7f8cb21e8223"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:41.969712 18467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:54:41.980157 18467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:54:41.983385 18467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:54:41.983605 18467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:54:41.991848 18467 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:42343
I20260620 14:54:41.991859 18534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:42343 every 8 connection(s)
I20260620 14:54:41.992936 18467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:54:41.995780 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 18467
I20260620 14:54:41.995909 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:54:41.995929 18535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:54:41.996201 7267 external_mini_cluster.cc:1428] Setting key 353898e06a386af37b7d5d98374d6b29
I20260620 14:54:42.003273 18535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc: Bootstrap starting.
I20260620 14:54:42.005879 18535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc: Neither blocks nor log segments found. Creating new log.
I20260620 14:54:42.006776 18535 log.cc:824] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc: Log is configured to *not* fsync() on all Append() calls
Jun 20 14:54:42 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:54:42.009146 18535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc: No bootstrap required, opened a new log
I20260620 14:54:42.012923 18535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1bc7be0044ebd93e84450cbef62fc" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42343 } }
I20260620 14:54:42.013294 18535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:54:42.013372 18535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab1bc7be0044ebd93e84450cbef62fc, State: Initialized, Role: FOLLOWER
I20260620 14:54:42.013850 18535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1bc7be0044ebd93e84450cbef62fc" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42343 } }
I20260620 14:54:42.014005 18535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:54:42.014099 18535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:54:42.014227 18535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:54:42.014240 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:41.997714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.7.24.254:42343)
0620 14:54:41.998197 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0620 14:54:41.998215 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 14:54:41.998258 (+ 43us) server_negotiation.cc:407] Connection header received
0620 14:54:41.999429 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:54:41.999455 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:54:41.999899 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:54:42.000270 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:54:42.001130 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:42.002675 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:42.003926 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:42.004610 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:42.004766 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:42.007881 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:54:42.007888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:54:42.007907 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:54:42.007947 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:54:42.010793 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:54:42.011371 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:54:42.011378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:54:42.011385 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:54:42.011524 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:54:42.011848 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:54:42.011851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:54:42.011853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:54:42.012406 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:54:42.012606 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0620 14:54:42.012929 (+ 323us) server_negotiation.cc:299] Negotiation successful
0620 14:54:42.013236 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":152,"threads_started":1}
I20260620 14:54:42.015508 18535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1bc7be0044ebd93e84450cbef62fc" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42343 } }
I20260620 14:54:42.015849 18535 leader_election.cc:302] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ab1bc7be0044ebd93e84450cbef62fc; no voters:
I20260620 14:54:42.016204 18535 leader_election.cc:288] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:54:42.016278 18540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:54:42.016559 18540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [term 1 LEADER]: Becoming Leader. State: Replica: 7ab1bc7be0044ebd93e84450cbef62fc, State: Running, Role: LEADER
I20260620 14:54:42.016907 18540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1bc7be0044ebd93e84450cbef62fc" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42343 } }
I20260620 14:54:42.017678 18535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:54:42.018297 18540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ab1bc7be0044ebd93e84450cbef62fc. Latest consensus state: current_term: 1 leader_uuid: "7ab1bc7be0044ebd93e84450cbef62fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1bc7be0044ebd93e84450cbef62fc" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42343 } } }
I20260620 14:54:42.018539 18540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [sys.catalog]: This master's current role is: LEADER
I20260620 14:54:42.018798 18540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ab1bc7be0044ebd93e84450cbef62fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab1bc7be0044ebd93e84450cbef62fc" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42343 } } }
I20260620 14:54:42.018934 18540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc [sys.catalog]: This master's current role is: LEADER
I20260620 14:54:42.019273 18547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:54:42.022893 18547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:54:42.023329 18535 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 14:54:42.034037 18547 catalog_manager.cc:1347] Generated new cluster ID: f6f848a44091487aa187c14cb55ad8cb
I20260620 14:54:42.034170 18547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:54:42.052556 18547 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:54:42.054395 18547 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:54:42.061601 18547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc: Generated new TSK 0
I20260620 14:54:42.062505 18547 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:54:42.697469 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:41869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42343
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:54:42.838645 18580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:54:42.839064 18580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:54:42.839195 18580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:54:42.845726 18580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:54:42.845899 18580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:54:42.846047 18580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:54:42.852115 18580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.24.212:41869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:54:42.853312 18580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:54:42.854363 18580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:54:42.864405 18587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:54:42.864380 18590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:54:42.864372 18588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:54:42.864991 18580 server_base.cc:1034] running on GCE node
I20260620 14:54:42.865463 18580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:54:42.866223 18580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:54:42.867405 18580 hybrid_clock.cc:648] HybridClock initialized: now 1781967282867342 us; error 81 us; skew 500 ppm
Jun 20 14:54:42 dist-test-slave-sms9 krb5kdc[18023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967282, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:54:42.870255 18580 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:54:42.871282 18580 webserver.cc:460] Webserver started at http://127.7.24.193:35475/ using document root <none> and password file <none>
I20260620 14:54:42.871965 18580 fs_manager.cc:359] Metadata directory not provided
I20260620 14:54:42.872061 18580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:54:42.872337 18580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 14:54:42 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967282, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:54:42.893090 18580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "756da8fcecdd4acaae02fbb13f5c171b"
format_stamp: "Formatted at 2026-06-20 14:54:42 on dist-test-slave-sms9"
server_key: "43511e2e35dbca45941b368fc48868a0"
server_key_iv: "2b478fe5508a4c82fa1c8faf7934c9c6"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:42.893887 18580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "756da8fcecdd4acaae02fbb13f5c171b"
format_stamp: "Formatted at 2026-06-20 14:54:42 on dist-test-slave-sms9"
server_key: "43511e2e35dbca45941b368fc48868a0"
server_key_iv: "2b478fe5508a4c82fa1c8faf7934c9c6"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:42.899256 18580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260620 14:54:42.919765 18597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:54:42.921680 18580 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260620 14:54:42.921837 18580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "756da8fcecdd4acaae02fbb13f5c171b"
format_stamp: "Formatted at 2026-06-20 14:54:42 on dist-test-slave-sms9"
server_key: "43511e2e35dbca45941b368fc48868a0"
server_key_iv: "2b478fe5508a4c82fa1c8faf7934c9c6"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:42.921990 18580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:54:42.939972 18580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:54:42.943764 18580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:54:42.944016 18580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:54:42.944808 18580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:54:42.945909 18580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:54:42.946012 18580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:54:42.946206 18580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:54:42.946266 18580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:54:42.957923 18580 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:40361
I20260620 14:54:42.957975 18711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:40361 every 8 connection(s)
I20260620 14:54:42.958926 18580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:54:42.966612 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 18580
I20260620 14:54:42.966756 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:54:42.967051 7267 external_mini_cluster.cc:1428] Setting key 697b34041ff1e06fbe311ca5eea2428a
Jun 20 14:54:42 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967282, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:54:42.976976 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:42.961245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42343 (local address 127.7.24.193:55481)
0620 14:54:42.961710 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0620 14:54:42.961763 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 14:54:42.962653 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:54:42.963382 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:54:42.963389 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:54:42.963659 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:54:42.964142 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:42.964155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:42.965122 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:42.965126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:42.966081 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:42.966091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:42.966785 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:42.966790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:42.967002 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:42.967765 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:54:42.967788 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:54:42.971044 (+ 3256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:54:42.973431 (+ 2387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:54:42.973438 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:54:42.973456 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:54:42.973835 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:54:42.974296 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:54:42.974299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:54:42.974301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:54:42.974509 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:54:42.974996 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:54:42.975003 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:54:42.975333 (+ 330us) client_negotiation.cc:769] Sending connection context
0620 14:54:42.975558 (+ 225us) client_negotiation.cc:240] Negotiation successful
0620 14:54:42.975917 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":120,"threads_started":1}
I20260620 14:54:42.978341 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:42.960850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:55481 (local address 127.7.24.254:42343)
0620 14:54:42.961182 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0620 14:54:42.961191 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 14:54:42.961881 (+ 690us) server_negotiation.cc:407] Connection header received
0620 14:54:42.962916 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:54:42.963014 (+ 98us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:54:42.963080 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:54:42.963205 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:54:42.964294 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:42.964984 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:42.966205 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:42.966602 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:42.966776 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:42.971255 (+ 4479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:54:42.971259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:54:42.971262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:54:42.971290 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:54:42.973293 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:54:42.974025 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:54:42.974028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:54:42.974030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:54:42.974141 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:54:42.974644 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:54:42.974647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:54:42.974649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:54:42.974870 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:54:42.977952 (+ 3082us) server_negotiation.cc:1035] Waiting for connection context
0620 14:54:42.978031 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 14:54:42.978170 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":117,"threads_started":1}
I20260620 14:54:42.979704 18712 heartbeater.cc:347] Connected to a master server at 127.7.24.254:42343
I20260620 14:54:42.979941 18712 heartbeater.cc:464] Registering TS with master...
I20260620 14:54:42.980499 18712 heartbeater.cc:511] Master 127.7.24.254:42343 requested a full tablet report, sending...
I20260620 14:54:42.982254 18499 ts_manager.cc:194] Registered new tserver with Master: 756da8fcecdd4acaae02fbb13f5c171b (127.7.24.193:40361)
I20260620 14:54:42.983843 18499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:55481
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:54:43.019171 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:41869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42343
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 14:54:43 dist-test-slave-sms9 krb5kdc[18023](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781967283, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 14:54:43.155592 18719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:54:43.155970 18719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:54:43.156039 18719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:54:43.162137 18719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:54:43.162227 18719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:54:43.162317 18719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:54:43.170548 18719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.24.212:41869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:54:43.172014 18719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:54:43.173355 18719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:54:43.185851 18725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:54:43.186216 18728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:54:43.186862 18726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:54:43.187537 18719 server_base.cc:1034] running on GCE node
I20260620 14:54:43.188073 18719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:54:43.188932 18719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:54:43.190145 18719 hybrid_clock.cc:648] HybridClock initialized: now 1781967283190111 us; error 43 us; skew 500 ppm
Jun 20 14:54:43 dist-test-slave-sms9 krb5kdc[18023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967283, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:54:43.193876 18719 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:54:43.195034 18719 webserver.cc:460] Webserver started at http://127.7.24.194:32781/ using document root <none> and password file <none>
I20260620 14:54:43.195793 18719 fs_manager.cc:359] Metadata directory not provided
I20260620 14:54:43.195871 18719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:54:43.196105 18719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 14:54:43 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967283, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:54:43.218600 18719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "fbff7af0109541e983b171a6b9b0be38"
format_stamp: "Formatted at 2026-06-20 14:54:43 on dist-test-slave-sms9"
server_key: "c98f2447c65b24518d6e3c6f3db705cf"
server_key_iv: "80db32d69140b70582b605998e0ecbdd"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:43.219663 18719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "fbff7af0109541e983b171a6b9b0be38"
format_stamp: "Formatted at 2026-06-20 14:54:43 on dist-test-slave-sms9"
server_key: "c98f2447c65b24518d6e3c6f3db705cf"
server_key_iv: "80db32d69140b70582b605998e0ecbdd"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:43.225144 18719 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 14:54:43.242841 18735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:54:43.244470 18719 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260620 14:54:43.244625 18719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "fbff7af0109541e983b171a6b9b0be38"
format_stamp: "Formatted at 2026-06-20 14:54:43 on dist-test-slave-sms9"
server_key: "c98f2447c65b24518d6e3c6f3db705cf"
server_key_iv: "80db32d69140b70582b605998e0ecbdd"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:43.244772 18719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:54:43.264429 18719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:54:43.268457 18719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:54:43.268672 18719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:54:43.269349 18719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:54:43.270372 18719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:54:43.270433 18719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:54:43.270509 18719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:54:43.270538 18719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:54:43.281324 18719 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:34685
I20260620 14:54:43.281497 18848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:34685 every 8 connection(s)
I20260620 14:54:43.282459 18719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:54:43.287756 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 18719
I20260620 14:54:43.287895 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:54:43.288209 7267 external_mini_cluster.cc:1428] Setting key e3a50e6dec710e7ba7441645179d2fe5
Jun 20 14:54:43 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967283, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:54:43.300549 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.284747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:36169 (local address 127.7.24.254:42343)
0620 14:54:43.284900 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 14:54:43.284906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:54:43.286701 (+ 1795us) server_negotiation.cc:407] Connection header received
0620 14:54:43.287943 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:54:43.287947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:54:43.288010 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:54:43.288071 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:54:43.289744 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.290608 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.292358 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.292748 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.292834 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.296002 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:54:43.296006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:54:43.296009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:54:43.296038 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:54:43.297919 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:54:43.298673 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:54:43.298677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:54:43.298680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:54:43.298773 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:54:43.299240 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:54:43.299246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:54:43.299249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:54:43.299516 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:54:43.299625 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 14:54:43.300302 (+ 677us) server_negotiation.cc:299] Negotiation successful
0620 14:54:43.300418 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 14:54:43.301766 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.285274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42343 (local address 127.7.24.194:36169)
0620 14:54:43.286504 (+ 1230us) negotiation.cc:107] Waiting for socket to connect
0620 14:54:43.286572 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0620 14:54:43.287684 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:54:43.288488 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:54:43.288498 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:54:43.288867 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:54:43.289551 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.289586 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.290775 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.290779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.292209 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.292221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.293052 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.293057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.293268 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.293997 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:54:43.294019 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:54:43.295787 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:54:43.298077 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:54:43.298084 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:54:43.298103 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:54:43.298495 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:54:43.298904 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:54:43.298906 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:54:43.298908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:54:43.299091 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:54:43.299655 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:54:43.299664 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:54:43.300010 (+ 346us) client_negotiation.cc:769] Sending connection context
0620 14:54:43.300224 (+ 214us) client_negotiation.cc:240] Negotiation successful
0620 14:54:43.300586 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"thread_start_us":164,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
I20260620 14:54:43.303124 18849 heartbeater.cc:347] Connected to a master server at 127.7.24.254:42343
I20260620 14:54:43.303395 18849 heartbeater.cc:464] Registering TS with master...
I20260620 14:54:43.303982 18849 heartbeater.cc:511] Master 127.7.24.254:42343 requested a full tablet report, sending...
I20260620 14:54:43.305110 18499 ts_manager.cc:194] Registered new tserver with Master: fbff7af0109541e983b171a6b9b0be38 (127.7.24.194:34685)
I20260620 14:54:43.305703 18499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:36169
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:54:43.322299 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:41869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42343
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 14:54:43 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967283, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
W20260620 14:54:43.479085 18854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:54:43.479445 18854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:54:43.479514 18854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:54:43.485713 18854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:54:43.485818 18854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:54:43.485908 18854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:54:43.493839 18854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.24.212:41869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:42343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:54:43.495689 18854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:54:43.497030 18854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:54:43.510481 18859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:54:43.510607 18860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:54:43.510363 18862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:54:43.511132 18854 server_base.cc:1034] running on GCE node
I20260620 14:54:43.511718 18854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:54:43.512656 18854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:54:43.513829 18854 hybrid_clock.cc:648] HybridClock initialized: now 1781967283513790 us; error 48 us; skew 500 ppm
Jun 20 14:54:43 dist-test-slave-sms9 krb5kdc[18023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967283, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:54:43.517603 18854 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:54:43.519009 18854 webserver.cc:460] Webserver started at http://127.7.24.195:35855/ using document root <none> and password file <none>
I20260620 14:54:43.519842 18854 fs_manager.cc:359] Metadata directory not provided
I20260620 14:54:43.519990 18854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:54:43.520292 18854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 14:54:43 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967283, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:54:43.536015 18854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "eca92c27161a437b826de328739cc5a8"
format_stamp: "Formatted at 2026-06-20 14:54:43 on dist-test-slave-sms9"
server_key: "d15f7df343da5ac0726a8917d727e84e"
server_key_iv: "0c71629d3fdae356eea780180166e1c1"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:43.536778 18854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "eca92c27161a437b826de328739cc5a8"
format_stamp: "Formatted at 2026-06-20 14:54:43 on dist-test-slave-sms9"
server_key: "d15f7df343da5ac0726a8917d727e84e"
server_key_iv: "0c71629d3fdae356eea780180166e1c1"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:43.544030 18854 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260620 14:54:43.570214 18869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:54:43.571895 18854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:54:43.572024 18854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "eca92c27161a437b826de328739cc5a8"
format_stamp: "Formatted at 2026-06-20 14:54:43 on dist-test-slave-sms9"
server_key: "d15f7df343da5ac0726a8917d727e84e"
server_key_iv: "0c71629d3fdae356eea780180166e1c1"
server_key_version: "kuduclusterkey@0"
I20260620 14:54:43.572160 18854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:54:43.591025 18854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:54:43.594834 18854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:54:43.595098 18854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:54:43.595851 18854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:54:43.596947 18854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:54:43.597050 18854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:54:43.597160 18854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:54:43.597224 18854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:54:43.611527 18854 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:40849
I20260620 14:54:43.612910 18854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:54:43.618223 18982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:40849 every 8 connection(s)
I20260620 14:54:43.621803 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 18854
I20260620 14:54:43.621928 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:54:43.622218 7267 external_mini_cluster.cc:1428] Setting key fb7557d969f070ea5840a33dfd0dc264
Jun 20 14:54:43 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967283, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:54:43.641079 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.623297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:39215 (local address 127.7.24.254:42343)
0620 14:54:43.623656 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0620 14:54:43.623661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:54:43.625691 (+ 2030us) server_negotiation.cc:407] Connection header received
0620 14:54:43.627094 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:54:43.627098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:54:43.627153 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:54:43.627224 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:54:43.628955 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.629814 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.631566 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.631905 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.631984 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.635184 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:54:43.635187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:54:43.635190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:54:43.635217 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:54:43.638680 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:54:43.639331 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:54:43.639335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:54:43.639337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:54:43.639421 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:54:43.639804 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:54:43.639807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:54:43.639809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:54:43.640022 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:54:43.640112 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 14:54:43.640771 (+ 659us) server_negotiation.cc:299] Negotiation successful
0620 14:54:43.640900 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276}
I20260620 14:54:43.642967 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.621301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42343 (local address 127.7.24.195:39215)
0620 14:54:43.625533 (+ 4232us) negotiation.cc:107] Waiting for socket to connect
0620 14:54:43.625600 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0620 14:54:43.626815 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:54:43.627728 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:54:43.627738 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:54:43.628119 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:54:43.628790 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.628810 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.629973 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.629977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.631429 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.631439 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.632214 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.632218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.632415 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.633219 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:54:43.633241 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:54:43.634966 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:54:43.638826 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:54:43.638832 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:54:43.638851 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:54:43.639216 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:54:43.639523 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:54:43.639526 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:54:43.639528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:54:43.639702 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:54:43.640177 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:54:43.640184 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:54:43.640499 (+ 315us) client_negotiation.cc:769] Sending connection context
0620 14:54:43.641533 (+ 1034us) client_negotiation.cc:240] Negotiation successful
0620 14:54:43.641884 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3993,"thread_start_us":169,"threads_started":1}
I20260620 14:54:43.644469 18983 heartbeater.cc:347] Connected to a master server at 127.7.24.254:42343
I20260620 14:54:43.644704 18983 heartbeater.cc:464] Registering TS with master...
I20260620 14:54:43.645259 18983 heartbeater.cc:511] Master 127.7.24.254:42343 requested a full tablet report, sending...
I20260620 14:54:43.646373 18499 ts_manager.cc:194] Registered new tserver with Master: eca92c27161a437b826de328739cc5a8 (127.7.24.195:40849)
I20260620 14:54:43.647135 18499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:39215
I20260620 14:54:43.650663 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:54:43.663657 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.653411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46132 (local address 127.7.24.254:42343)
0620 14:54:43.653515 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 14:54:43.653519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:54:43.654067 (+ 548us) server_negotiation.cc:407] Connection header received
0620 14:54:43.654244 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:54:43.654247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:54:43.654290 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:54:43.654338 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:54:43.655334 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.656195 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.657405 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.657762 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.657840 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.659685 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:54:43.659689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:54:43.659691 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:54:43.659717 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:54:43.661456 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:54:43.661939 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:54:43.661943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:54:43.661945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:54:43.662023 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:54:43.662331 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:54:43.662335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:54:43.662337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:54:43.662606 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:54:43.662712 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 14:54:43.663288 (+ 576us) server_negotiation.cc:299] Negotiation successful
0620 14:54:43.663410 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 14:54:43.667867 18499 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:54:43.672676 18499 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:54:43.696138 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.688080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:40849 (local address 127.0.0.1:38310)
0620 14:54:43.690549 (+ 2469us) negotiation.cc:107] Waiting for socket to connect
0620 14:54:43.690570 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 14:54:43.690649 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:54:43.691229 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:54:43.691232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:54:43.691251 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:54:43.691372 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.691383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.692761 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.692764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.694121 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.694132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.695497 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.695503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.695642 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.695683 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 14:54:43.695788 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 14:54:43.695915 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2412,"mutex_wait_us":1,"spinlock_wait_cycles":896,"thread_start_us":81,"threads_started":1}
I20260620 14:54:43.696645 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.688459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38310 (local address 127.7.24.195:40849)
0620 14:54:43.689840 (+ 1381us) server_negotiation.cc:206] Beginning negotiation
0620 14:54:43.689853 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 14:54:43.690734 (+ 881us) server_negotiation.cc:407] Connection header received
0620 14:54:43.690810 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:54:43.690815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:54:43.690970 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:54:43.691094 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:54:43.691508 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.692637 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.694275 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.695133 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.695234 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.695435 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0620 14:54:43.696348 (+ 913us) server_negotiation.cc:299] Negotiation successful
0620 14:54:43.696475 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1294,"thread_start_us":145,"threads_started":1}
I20260620 14:54:43.697798 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.684936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:40361 (local address 127.0.0.1:54788)
0620 14:54:43.690120 (+ 5184us) negotiation.cc:107] Waiting for socket to connect
0620 14:54:43.690181 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0620 14:54:43.690323 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:54:43.691174 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:54:43.691180 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:54:43.691204 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:54:43.692199 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.692209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.693541 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.693544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.695521 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.695528 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.697395 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.697399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.697498 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.697522 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 14:54:43.697596 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 14:54:43.697669 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4980,"mutex_wait_us":88,"spinlock_wait_cycles":10240,"thread_start_us":122,"threads_started":1}
I20260620 14:54:43.698307 18997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.685882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.7.24.193:40361)
0620 14:54:43.689811 (+ 3929us) server_negotiation.cc:206] Beginning negotiation
0620 14:54:43.689820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 14:54:43.690491 (+ 671us) server_negotiation.cc:407] Connection header received
0620 14:54:43.690583 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:54:43.690593 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:54:43.690835 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:54:43.690972 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:54:43.692331 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.693408 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.695652 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.697108 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.697207 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.697333 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 14:54:43.698070 (+ 737us) server_negotiation.cc:299] Negotiation successful
0620 14:54:43.698176 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3805,"thread_start_us":98,"threads_started":1}
I20260620 14:54:43.699900 18916 tablet_service.cc:1467] Processing CreateTablet for tablet e7ef38a5931649d982d1859451e57907 (DEFAULT_TABLE table=test-table [id=ccd7d112c484475bb816225ee10f37b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:54:43.701063 18916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7ef38a5931649d982d1859451e57907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:54:43.702152 18645 tablet_service.cc:1467] Processing CreateTablet for tablet e7ef38a5931649d982d1859451e57907 (DEFAULT_TABLE table=test-table [id=ccd7d112c484475bb816225ee10f37b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:54:43.703178 18645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7ef38a5931649d982d1859451e57907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:54:43.703326 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.687472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:34685 (local address 127.0.0.1:33550)
0620 14:54:43.690120 (+ 2648us) negotiation.cc:107] Waiting for socket to connect
0620 14:54:43.690181 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0620 14:54:43.690322 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:54:43.699166 (+ 8844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:54:43.699170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:54:43.699190 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:54:43.699273 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.699279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.700647 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.700651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.701777 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.701788 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.702943 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.702947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.703042 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.703069 (+ 27us) client_negotiation.cc:769] Sending connection context
0620 14:54:43.703116 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 14:54:43.703184 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2478,"spinlock_wait_cycles":3200,"thread_start_us":83,"threads_started":1}
I20260620 14:54:43.704047 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.687683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33550 (local address 127.7.24.194:34685)
0620 14:54:43.698507 (+ 10824us) server_negotiation.cc:206] Beginning negotiation
0620 14:54:43.698519 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:54:43.698547 (+ 28us) server_negotiation.cc:407] Connection header received
0620 14:54:43.698636 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:54:43.698642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:54:43.698857 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:54:43.699015 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:54:43.699391 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.700491 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.701950 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.702807 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.703616 (+ 809us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.703768 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 14:54:43.703829 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 14:54:43.703917 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10419,"thread_start_us":103,"threads_started":1}
I20260620 14:54:43.707115 18783 tablet_service.cc:1467] Processing CreateTablet for tablet e7ef38a5931649d982d1859451e57907 (DEFAULT_TABLE table=test-table [id=ccd7d112c484475bb816225ee10f37b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:54:43.708178 18783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7ef38a5931649d982d1859451e57907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:54:43.714990 19002 tablet_bootstrap.cc:492] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b: Bootstrap starting.
I20260620 14:54:43.718896 19002 tablet_bootstrap.cc:654] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b: Neither blocks nor log segments found. Creating new log.
I20260620 14:54:43.720074 19002 log.cc:824] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b: Log is configured to *not* fsync() on all Append() calls
I20260620 14:54:43.721019 19004 tablet_bootstrap.cc:492] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8: Bootstrap starting.
I20260620 14:54:43.721661 19003 tablet_bootstrap.cc:492] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: Bootstrap starting.
I20260620 14:54:43.723949 19004 tablet_bootstrap.cc:654] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8: Neither blocks nor log segments found. Creating new log.
I20260620 14:54:43.724284 19003 tablet_bootstrap.cc:654] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: Neither blocks nor log segments found. Creating new log.
I20260620 14:54:43.724880 19004 log.cc:824] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8: Log is configured to *not* fsync() on all Append() calls
I20260620 14:54:43.726229 19003 log.cc:824] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: Log is configured to *not* fsync() on all Append() calls
I20260620 14:54:43.727114 19004 tablet_bootstrap.cc:492] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8: No bootstrap required, opened a new log
I20260620 14:54:43.727329 19004 ts_tablet_manager.cc:1397] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260620 14:54:43.728662 19002 tablet_bootstrap.cc:492] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b: No bootstrap required, opened a new log
I20260620 14:54:43.728928 19002 ts_tablet_manager.cc:1397] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260620 14:54:43.731559 19004 raft_consensus.cc:348] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756da8fcecdd4acaae02fbb13f5c171b" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 40361 } } peers { permanent_uuid: "fbff7af0109541e983b171a6b9b0be38" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34685 } } peers { permanent_uuid: "eca92c27161a437b826de328739cc5a8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 40849 } }
I20260620 14:54:43.731839 19004 raft_consensus.cc:374] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:54:43.731891 19004 raft_consensus.cc:729] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca92c27161a437b826de328739cc5a8, State: Initialized, Role: FOLLOWER
I20260620 14:54:43.732388 19004 consensus_queue.cc:260] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756da8fcecdd4acaae02fbb13f5c171b" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 40361 } } peers { permanent_uuid: "fbff7af0109541e983b171a6b9b0be38" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34685 } } peers { permanent_uuid: "eca92c27161a437b826de328739cc5a8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 40849 } }
I20260620 14:54:43.733409 19004 ts_tablet_manager.cc:1428] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260620 14:54:43.733800 18983 heartbeater.cc:503] Master 127.7.24.254:42343 was elected leader, sending a full tablet report...
I20260620 14:54:43.734030 19002 raft_consensus.cc:348] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756da8fcecdd4acaae02fbb13f5c171b" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 40361 } } peers { permanent_uuid: "fbff7af0109541e983b171a6b9b0be38" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34685 } } peers { permanent_uuid: "eca92c27161a437b826de328739cc5a8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 40849 } }
I20260620 14:54:43.734359 19002 raft_consensus.cc:374] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:54:43.734479 19002 raft_consensus.cc:729] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756da8fcecdd4acaae02fbb13f5c171b, State: Initialized, Role: FOLLOWER
I20260620 14:54:43.734975 19002 consensus_queue.cc:260] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756da8fcecdd4acaae02fbb13f5c171b" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 40361 } } peers { permanent_uuid: "fbff7af0109541e983b171a6b9b0be38" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34685 } } peers { permanent_uuid: "eca92c27161a437b826de328739cc5a8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 40849 } }
I20260620 14:54:43.736547 19002 ts_tablet_manager.cc:1428] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20260620 14:54:43.737072 18712 heartbeater.cc:503] Master 127.7.24.254:42343 was elected leader, sending a full tablet report...
I20260620 14:54:43.738880 19003 tablet_bootstrap.cc:492] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: No bootstrap required, opened a new log
I20260620 14:54:43.739126 19003 ts_tablet_manager.cc:1397] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.004s
I20260620 14:54:43.743202 19003 raft_consensus.cc:348] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756da8fcecdd4acaae02fbb13f5c171b" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 40361 } } peers { permanent_uuid: "fbff7af0109541e983b171a6b9b0be38" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34685 } } peers { permanent_uuid: "eca92c27161a437b826de328739cc5a8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 40849 } }
I20260620 14:54:43.743462 19003 raft_consensus.cc:374] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:54:43.743552 19003 raft_consensus.cc:729] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbff7af0109541e983b171a6b9b0be38, State: Initialized, Role: FOLLOWER
I20260620 14:54:43.744024 19003 consensus_queue.cc:260] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756da8fcecdd4acaae02fbb13f5c171b" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 40361 } } peers { permanent_uuid: "fbff7af0109541e983b171a6b9b0be38" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34685 } } peers { permanent_uuid: "eca92c27161a437b826de328739cc5a8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 40849 } }
I20260620 14:54:43.744657 18849 heartbeater.cc:503] Master 127.7.24.254:42343 was elected leader, sending a full tablet report...
I20260620 14:54:43.744987 19003 ts_tablet_manager.cc:1428] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260620 14:54:43.785866 18850 tablet.cc:2369] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 14:54:43 dist-test-slave-sms9 krb5kdc[18023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967283, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:54:43.833216 19008 raft_consensus.cc:482] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:54:43.833451 19008 raft_consensus.cc:504] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756da8fcecdd4acaae02fbb13f5c171b" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 40361 } } peers { permanent_uuid: "fbff7af0109541e983b171a6b9b0be38" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34685 } } peers { permanent_uuid: "eca92c27161a437b826de328739cc5a8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 40849 } }
I20260620 14:54:43.834920 19008 leader_election.cc:288] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 756da8fcecdd4acaae02fbb13f5c171b (127.7.24.193:40361), fbff7af0109541e983b171a6b9b0be38 (127.7.24.194:34685)
I20260620 14:54:43.841794 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.835266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:40361 (local address 127.7.24.195:49061)
0620 14:54:43.835553 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0620 14:54:43.835568 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:54:43.835666 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:54:43.836226 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:54:43.836229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:54:43.836248 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:54:43.836359 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.836365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.839046 (+ 2681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.839050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.840285 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.840298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.841316 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.841320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.841435 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.841459 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 14:54:43.841543 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 14:54:43.841636 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260620 14:54:43.843173 18997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.835814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:49061 (local address 127.7.24.193:40361)
0620 14:54:43.835928 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 14:54:43.835933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:54:43.835946 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:54:43.835998 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:54:43.836001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:54:43.836045 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:54:43.836111 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:54:43.838150 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.838916 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.840442 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.841055 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.841179 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.841236 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 14:54:43.842947 (+ 1711us) server_negotiation.cc:299] Negotiation successful
0620 14:54:43.843040 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:54:43.844461 18666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7ef38a5931649d982d1859451e57907" candidate_uuid: "eca92c27161a437b826de328739cc5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756da8fcecdd4acaae02fbb13f5c171b" is_pre_election: true
I20260620 14:54:43.844825 18666 raft_consensus.cc:2454] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca92c27161a437b826de328739cc5a8 in term 0.
I20260620 14:54:43.845461 18870 leader_election.cc:302] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 756da8fcecdd4acaae02fbb13f5c171b, eca92c27161a437b826de328739cc5a8; no voters:
I20260620 14:54:43.845983 19008 raft_consensus.cc:2781] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:54:43.846071 19008 raft_consensus.cc:482] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:54:43.846108 19008 raft_consensus.cc:3037] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:54:43.847698 19008 raft_consensus.cc:504] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756da8fcecdd4acaae02fbb13f5c171b" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 40361 } } peers { permanent_uuid: "fbff7af0109541e983b171a6b9b0be38" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34685 } } peers { permanent_uuid: "eca92c27161a437b826de328739cc5a8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 40849 } }
I20260620 14:54:43.849695 18666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7ef38a5931649d982d1859451e57907" candidate_uuid: "eca92c27161a437b826de328739cc5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756da8fcecdd4acaae02fbb13f5c171b"
I20260620 14:54:43.849823 18666 raft_consensus.cc:3037] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:54:43.851096 18666 raft_consensus.cc:2454] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca92c27161a437b826de328739cc5a8 in term 1.
I20260620 14:54:43.851701 18870 leader_election.cc:302] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 756da8fcecdd4acaae02fbb13f5c171b, eca92c27161a437b826de328739cc5a8; no voters:
I20260620 14:54:43.852010 19008 leader_election.cc:288] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 756da8fcecdd4acaae02fbb13f5c171b (127.7.24.193:40361), fbff7af0109541e983b171a6b9b0be38 (127.7.24.194:34685)
I20260620 14:54:43.852082 19014 raft_consensus.cc:2781] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:54:43.852202 19014 raft_consensus.cc:686] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 1 LEADER]: Becoming Leader. State: Replica: eca92c27161a437b826de328739cc5a8, State: Running, Role: LEADER
I20260620 14:54:43.852710 19014 consensus_queue.cc:237] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756da8fcecdd4acaae02fbb13f5c171b" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 40361 } } peers { permanent_uuid: "fbff7af0109541e983b171a6b9b0be38" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34685 } } peers { permanent_uuid: "eca92c27161a437b826de328739cc5a8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 40849 } }
I20260620 14:54:43.856367 18499 catalog_manager.cc:5495] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to eca92c27161a437b826de328739cc5a8 (127.7.24.195). New cstate: current_term: 1 leader_uuid: "eca92c27161a437b826de328739cc5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "756da8fcecdd4acaae02fbb13f5c171b" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 40361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbff7af0109541e983b171a6b9b0be38" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca92c27161a437b826de328739cc5a8" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 40849 } health_report { overall_health: HEALTHY } } }
I20260620 14:54:43.857906 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.836623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:34685 (local address 127.7.24.195:35359)
0620 14:54:43.841965 (+ 5342us) negotiation.cc:107] Waiting for socket to connect
0620 14:54:43.841981 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:54:43.842081 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:54:43.842671 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:54:43.842674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:54:43.842690 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:54:43.842781 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.842794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.847894 (+ 5100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.847898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.853185 (+ 5287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:54:43.853197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.857484 (+ 4287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.857488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:54:43.857579 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.857604 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 14:54:43.857678 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 14:54:43.857756 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5286,"thread_start_us":121,"threads_started":1}
I20260620 14:54:43.858343 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.837198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:35359 (local address 127.7.24.194:34685)
0620 14:54:43.837443 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0620 14:54:43.837447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:54:43.842190 (+ 4743us) server_negotiation.cc:407] Connection header received
0620 14:54:43.842261 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:54:43.842266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:54:43.842333 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:54:43.842409 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:54:43.843081 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.843849 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.853343 (+ 9494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.853956 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.854071 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.854124 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 14:54:43.858142 (+ 4018us) server_negotiation.cc:299] Negotiation successful
0620 14:54:43.858234 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260620 14:54:43.860196 18802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7ef38a5931649d982d1859451e57907" candidate_uuid: "eca92c27161a437b826de328739cc5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbff7af0109541e983b171a6b9b0be38" is_pre_election: true
I20260620 14:54:43.860482 18802 raft_consensus.cc:2454] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca92c27161a437b826de328739cc5a8 in term 0.
I20260620 14:54:43.860821 18801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7ef38a5931649d982d1859451e57907" candidate_uuid: "eca92c27161a437b826de328739cc5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbff7af0109541e983b171a6b9b0be38"
I20260620 14:54:43.860918 18801 raft_consensus.cc:3037] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:54:43.862185 18801 raft_consensus.cc:2454] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca92c27161a437b826de328739cc5a8 in term 1.
W20260620 14:54:43.869367 18984 tablet.cc:2369] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:54:43.888572 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.883200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.7.24.195:40849)
0620 14:54:43.883362 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 14:54:43.883367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:54:43.883566 (+ 199us) server_negotiation.cc:407] Connection header received
0620 14:54:43.883662 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:54:43.883665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:54:43.883742 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:54:43.883851 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:54:43.884280 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.885029 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.886306 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:54:43.886670 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:54:43.886771 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:54:43.887199 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:54:43.887774 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:54:43.887893 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 14:54:43.888337 (+ 444us) server_negotiation.cc:299] Negotiation successful
0620 14:54:43.888402 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260620 14:54:43.903893 18666 raft_consensus.cc:1261] T e7ef38a5931649d982d1859451e57907 P 756da8fcecdd4acaae02fbb13f5c171b [term 1 FOLLOWER]: Refusing update from remote peer eca92c27161a437b826de328739cc5a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:54:43.903893 18801 raft_consensus.cc:1261] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38 [term 1 FOLLOWER]: Refusing update from remote peer eca92c27161a437b826de328739cc5a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:54:43.904682 19008 consensus_queue.cc:1035] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbff7af0109541e983b171a6b9b0be38" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 34685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:54:43.905114 19008 consensus_queue.cc:1035] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "756da8fcecdd4acaae02fbb13f5c171b" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 40361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:54:43.935811 19018 mvcc.cc:205] Tried to move back new op lower bound from 7298937994858336256 to 7298937994663944192. Current Snapshot: MvccSnapshot[applied={T|T < 7298937994858336256}]
I20260620 14:54:43.936069 19020 mvcc.cc:205] Tried to move back new op lower bound from 7298937994858336256 to 7298937994663944192. Current Snapshot: MvccSnapshot[applied={T|T < 7298937994858336256}]
I20260620 14:54:43.947605 18499 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:54:43.947814 18499 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:54:43.950785 18499 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ab1bc7be0044ebd93e84450cbef62fc: Sending DeleteTablet for 3 replicas of tablet e7ef38a5931649d982d1859451e57907
I20260620 14:54:43.952463 18645 tablet_service.cc:1514] Processing DeleteTablet for tablet e7ef38a5931649d982d1859451e57907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:54:43 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:54788
I20260620 14:54:43.952579 18916 tablet_service.cc:1514] Processing DeleteTablet for tablet e7ef38a5931649d982d1859451e57907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:54:43 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:38310
I20260620 14:54:43.953035 18783 tablet_service.cc:1514] Processing DeleteTablet for tablet e7ef38a5931649d982d1859451e57907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:54:43 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:33550
I20260620 14:54:43.953164 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 18580
I20260620 14:54:43.954663 19028 tablet_replica.cc:330] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: stopping tablet replica
I20260620 14:54:43.954993 19028 raft_consensus.cc:2229] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:54:43.955304 19028 raft_consensus.cc:2258] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:54:43.956842 19028 ts_tablet_manager.cc:1905] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:54:43.959645 19028 ts_tablet_manager.cc:1918] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:54:43.959753 19028 log.cc:1196] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/e7ef38a5931649d982d1859451e57907
I20260620 14:54:43.960072 19028 ts_tablet_manager.cc:1939] T e7ef38a5931649d982d1859451e57907 P fbff7af0109541e983b171a6b9b0be38: Deleting consensus metadata
I20260620 14:54:43.961516 18486 catalog_manager.cc:4841] TS fbff7af0109541e983b171a6b9b0be38 (127.7.24.194:34685): tablet e7ef38a5931649d982d1859451e57907 (table test-table [id=ccd7d112c484475bb816225ee10f37b9]) successfully deleted
W20260620 14:54:43.970577 18483 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:40361 (error 108)
I20260620 14:54:43.971540 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 18719
I20260620 14:54:43.974742 19026 tablet_replica.cc:330] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8: stopping tablet replica
I20260620 14:54:43.975248 19026 raft_consensus.cc:2229] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:54:43.975813 19026 raft_consensus.cc:2258] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:54:43.977681 19026 ts_tablet_manager.cc:1905] T e7ef38a5931649d982d1859451e57907 P eca92c27161a437b826de328739cc5a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:54:43.980605 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 18854
I20260620 14:54:43.983479 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:43.983178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:40361 (local address 127.0.0.1:54802)
0620 14:54:43.983299 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 14:54:43.983375 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:40361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 14:54:43.983805 18483 catalog_manager.cc:4584] TS 756da8fcecdd4acaae02fbb13f5c171b (127.7.24.193:40361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e7ef38a5931649d982d1859451e57907: Network error: Client connection negotiation failed: client connection to 127.7.24.193:40361: connect: Connection refused (error 111)
I20260620 14:54:44.005298 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:54:44.004994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:40849 (local address 127.0.0.1:38326)
0620 14:54:44.005134 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 14:54:44.005198 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.195:40849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 14:54:44.005900 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 18467
I20260620 14:54:44.014003 7267 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 14:54:44.349350 7267 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 14:54:44.349483 7267 mini_ranger.cc:66] Stopping Ranger...
I20260620 14:54:44.694768 7267 mini_ranger.cc:68] Stopped Ranger
2026-06-20 14:54:44.695 UTC [18053] LOG: received smart shutdown request
2026-06-20 14:54:44.700 UTC [18053] LOG: background worker "logical replication launcher" (PID 18062) exited with exit code 1
2026-06-20 14:54:44.701 UTC [18057] LOG: shutting down
2026-06-20 14:54:44.892 UTC [18053] LOG: database system is shut down
2026-06-20T14:54:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47644 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:54:45 dist-test-slave-sms9 krb5kdc[19038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:54:45 dist-test-slave-sms9 krb5kdc[19038](info): set up 2 sockets
Jun 20 14:54:45 dist-test-slave-sms9 krb5kdc[19038](info): commencing operation
krb5kdc: starting...
W20260620 14:54:46.066740 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:54:46 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:54:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:54:46Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.24.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.24.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/rangeradmin_127.7.24.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/rangeradmin_127.7.24.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.24.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.24.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/rangerlookup_127.7.24.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/rangerlookup_127.7.24.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.24.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.24.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/HTTP_127.7.24.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/HTTP_127.7.24.212@KRBTEST.COM.keytab.
I20260620 14:54:46.247370 7267 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskNGdDYa/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/postgres -l logfile start
2026-06-20 14:54:48.833 UTC [19068] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 14:54:48.833 UTC [19068] LOG: listening on IPv4 address "127.7.24.212", port 54521
2026-06-20 14:54:48.837 UTC [19068] LOG: listening on Unix socket "/tmp/.s.PGSQL.54521"
2026-06-20 14:54:48.873 UTC [19071] LOG: database system was shut down at 2026-06-20 14:54:47 UTC
2026-06-20 14:54:48.877 UTC [19068] LOG: database system is ready to accept connections
I20260620 14:54:49.906330 7267 mini_postgres.cc:96] Postgres bound to 54521
2026-06-20 14:54:49.912 UTC [19079] FATAL: database "slave" does not exist
127.7.24.212:54521 - accepting connections
I20260620 14:54:49.913187 7267 mini_ranger.cc:161] Starting Ranger...
I20260620 14:54:49.928261 7267 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 14:54:50.435979 7267 mini_ranger.cc:87] Created ranger Postgres database
I20260620 14:54:50.436049 7267 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-admin
2026-06-20 14:54:50,714 [I] DB FLAVOR :POSTGRES
2026-06-20 14:54:50,714 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 14:54:50,714 [I] Checking connection..
2026-06-20 14:54:50,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 14:54:50,965 [I] Checking connection passed.
2026-06-20 14:54:50,966 [I] --------- Verifying version history table ---------
2026-06-20 14:54:50,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 14:54:51,241 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 14:54:51,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 14:54:51,498 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 14:54:51,498 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 14:54:51,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 14:54:51.756 UTC [19143] WARNING: there is no transaction in progress
2026-06-20 14:54:51,763 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 14:54:51,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 14:54:51,999 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 14:54:51,999 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 14:54:52,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 14:54:52,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 14:54:52,493 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sms9.c.gcp-upstream.internal','N') ;"
2026-06-20 14:54:52,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 14:54:52,967 [I] Table x_portal_user does not exist in database ranger
2026-06-20 14:54:52,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 14:54:53,229 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 14:54:53,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 14:54:53,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 14:54:53,730 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 14:54:53,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 14:54:54.725 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.742 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.761 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.778 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.799 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.824 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.888 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.895 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.911 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.922 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.932 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.948 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.958 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.966 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.973 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:54.980 UTC [19251] WARNING: there is no transaction in progress
2026-06-20 14:54:55,668 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 14:54:55,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 14:54:55,917 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 14:54:55,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 14:54:56,157 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 14:54:56,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 14:54:56,414 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 14:54:56,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 14:54:56,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 14:54:56,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sms9.c.gcp-upstream.internal';"
2026-06-20 14:54:57,162 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 14:54:57,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 14:54:57,423 [I] DB_PATCHES have already been applied
I20260620 14:54:57.429093 7267 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-admin:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskNGdDYa/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 14:54:57.429188 7267 mini_ranger.cc:193] Using host: 127.7.24.212
I20260620 14:54:57.432677 7267 mini_ranger.cc:227] Ranger admin URL: http://127.7.24.212:58963
Jun 20, 2026 2:54:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 2:54:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 2:54:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 2:54:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 2:54:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 2:54:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 2:54:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 14:54:58.547824 7267 mini_ranger.cc:160] Time spent starting Ranger: real 8.635s user 0.005s sys 0.005s
Jun 20, 2026 2:55:19 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 2:55:19 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 20, 2026 2:55:19 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 2:55:19 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 2:55:20 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 14:55:21.095247 7267 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.24.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.24.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/rangerkms_127.7.24.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.24.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/rangerkms_127.7.24.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.24.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.24.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.24.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/HTTP_127.7.24.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.24.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/HTTP_127.7.24.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 14:55:21 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967321, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 14:55:21.184851 7267 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 14:55:21.195616 7267 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 14:55:22.107299 7267 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 14:55:22.107375 7267 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-kms
I20260620 14:55:22.107388 7267 mini_ranger_kms.cc:227] Using postgres at 127.7.24.212:54521
2026-06-20 14:55:22,276 [I] DB FLAVOR :POSTGRES
2026-06-20 14:55:22,276 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 14:55:22,276 [I] Checking connection
2026-06-20 14:55:22,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 14:55:22,506 [I] connection success
2026-06-20 14:55:22,506 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 14:55:22,506 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 14:55:22,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 14:55:22,738 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 14:55:22,738 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 14:55:22,738 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 14:55:22,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.24.212:54521/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 14:55:22,998 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 14:55:23.470589 7267 mini_ranger_kms.cc:310] Created kms service
I20260620 14:55:23.642498 7267 mini_ranger_kms.cc:326] Created kudu user
I20260620 14:55:23.688901 7267 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 14:55:23 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967321, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:24.090581 7267 mini_ranger_kms.cc:384] Added ranger policy
I20260620 14:55:24.090729 7267 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-kms:/tmp/dist-test-taskNGdDYa/build/debug/bin/postgresql.jar:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskNGdDYa/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 14:55:24.090754 7267 mini_ranger_kms.cc:240] Using host: 127.7.24.212
I20260620 14:55:24.095091 7267 mini_ranger_kms.cc:276] Ranger KMS PID: 19431
I20260620 14:55:24.095191 7267 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.24.212:48583
Jun 20, 2026 2:55:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 2:55:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 2:55:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 2:55:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 2:55:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 2:55:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 20 14:55:26 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781967326, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.24.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:26.410846 7267 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.226s user 0.005s sys 0.149s
I20260620 14:55:26.410977 7267 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 14:55:26.411068 7267 mini_ranger_kms.cc:401] 127.7.24.212:48583/kms/v1/keys
Jun 20 14:55:26 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967326, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.24.212@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
Jun 20 14:55:27 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967326, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.24.212@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
Jun 20 14:55:29 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:55:29.447693 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:35483
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:45767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:35483
--ranger_config_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:48583/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:29.569754 19482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:29.570029 19482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:29.570067 19482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:29.574322 19482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:55:29.574388 19482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:29.574410 19482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:29.574426 19482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:55:29.574577 19482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:55:29.580523 19482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:45767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.24.212:48583/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:35483
--ranger_config_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:35483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:29.581631 19482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:29.582724 19482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:29.592224 19490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:29.592224 19488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:29.592335 19487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:29.592383 19482 server_base.cc:1034] running on GCE node
I20260620 14:55:29.593209 19482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:29.594590 19482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:29.595786 19482 hybrid_clock.cc:648] HybridClock initialized: now 1781967329595757 us; error 42 us; skew 500 ppm
Jun 20 14:55:29 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967329, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:29.603462 19482 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:55:29.604530 19482 webserver.cc:460] Webserver started at http://127.7.24.254:33141/ using document root <none> and password file <none>
I20260620 14:55:29.605130 19482 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:29.605187 19482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:29.605417 19482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 14:55:29 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967329, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:29.652372 19482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "dcf929f6d9f84a3ca7081e42d42e4014"
format_stamp: "Formatted at 2026-06-20 14:55:29 on dist-test-slave-sms9"
server_key: "eafdaa750fd7313a179e7873e6e97411"
server_key_iv: "2738a86b4993d0b1c90c21b379e8b846"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:29.653069 19482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "dcf929f6d9f84a3ca7081e42d42e4014"
format_stamp: "Formatted at 2026-06-20 14:55:29 on dist-test-slave-sms9"
server_key: "eafdaa750fd7313a179e7873e6e97411"
server_key_iv: "2738a86b4993d0b1c90c21b379e8b846"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:29.657258 19482 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:55:29.671880 19497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:29.673336 19482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:55:29.673503 19482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "dcf929f6d9f84a3ca7081e42d42e4014"
format_stamp: "Formatted at 2026-06-20 14:55:29 on dist-test-slave-sms9"
server_key: "eafdaa750fd7313a179e7873e6e97411"
server_key_iv: "2738a86b4993d0b1c90c21b379e8b846"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:29.673650 19482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:29.697160 19482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:29.700495 19482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:29.700947 19482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:29.708976 19482 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:35483
I20260620 14:55:29.709002 19549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:35483 every 8 connection(s)
I20260620 14:55:29.710003 19482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:55:29.713465 19550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:29.715730 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 19482
I20260620 14:55:29.715893 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:55:29.716220 7267 external_mini_cluster.cc:1428] Setting key c0d7805f25fd1b103db45259ccc35e3b
I20260620 14:55:29.721890 19550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014: Bootstrap starting.
I20260620 14:55:29.724665 19550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014: Neither blocks nor log segments found. Creating new log.
Jun 20 14:55:29 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:29.725601 19550 log.cc:824] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:29.728188 19550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014: No bootstrap required, opened a new log
I20260620 14:55:29.732409 19553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:29.717731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52228 (local address 127.7.24.254:35483)
0620 14:55:29.718201 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:29.718215 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:29.718250 (+ 35us) server_negotiation.cc:407] Connection header received
0620 14:55:29.719001 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:29.719021 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:29.719344 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:29.719679 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:29.720491 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:29.721511 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:29.722545 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:29.723011 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:29.723186 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:29.725863 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:29.725868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:29.725882 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:29.725909 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:29.728676 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:29.729197 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:29.729204 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:29.729210 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:29.729339 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:29.729650 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:29.729653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:29.729655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:29.730178 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:29.730373 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:29.731134 (+ 761us) server_negotiation.cc:299] Negotiation successful
0620 14:55:29.731421 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":155,"threads_started":1}
I20260620 14:55:29.732256 19550 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } }
I20260620 14:55:29.732517 19550 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:29.732563 19550 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf929f6d9f84a3ca7081e42d42e4014, State: Initialized, Role: FOLLOWER
I20260620 14:55:29.732975 19550 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } }
I20260620 14:55:29.733134 19550 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:55:29.733203 19550 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:55:29.733300 19550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:29.734524 19550 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } }
I20260620 14:55:29.734898 19550 leader_election.cc:302] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcf929f6d9f84a3ca7081e42d42e4014; no voters:
I20260620 14:55:29.735387 19550 leader_election.cc:288] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:55:29.735507 19555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:55:29.735723 19555 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 1 LEADER]: Becoming Leader. State: Replica: dcf929f6d9f84a3ca7081e42d42e4014, State: Running, Role: LEADER
I20260620 14:55:29.736131 19555 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } }
I20260620 14:55:29.736630 19550 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:55:29.737794 19556 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } } }
I20260620 14:55:29.738020 19556 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:29.738313 19556 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcf929f6d9f84a3ca7081e42d42e4014. Latest consensus state: current_term: 1 leader_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } } }
I20260620 14:55:29.738495 19556 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:29.738926 19550 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 14:55:29.743285 19565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:55:29.747624 19565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:55:29.755033 19565 catalog_manager.cc:1347] Generated new cluster ID: 6267f642bf04489180f10b1c0a854eff
I20260620 14:55:29.755105 19565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:55:29.787400 19565 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:55:29.788789 19565 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:55:29.796669 19565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014: Generated new TSK 0
I20260620 14:55:29.797554 19565 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:55:30.510270 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:48583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35483
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:45767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:30.663081 19595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:30.663383 19595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:30.663511 19595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:30.668113 19595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:30.668206 19595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:30.668298 19595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:55:30.673786 19595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:45767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.24.212:48583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:30.674978 19595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:30.676014 19595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:30.686020 19605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:30.686065 19603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:30.686038 19602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:30.686554 19595 server_base.cc:1034] running on GCE node
I20260620 14:55:30.687037 19595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:30.687778 19595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:30.688987 19595 hybrid_clock.cc:648] HybridClock initialized: now 1781967330688954 us; error 43 us; skew 500 ppm
Jun 20 14:55:30 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:30.692811 19595 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:55:30.693889 19595 webserver.cc:460] Webserver started at http://127.7.24.193:36831/ using document root <none> and password file <none>
I20260620 14:55:30.694562 19595 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:30.694664 19595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:30.694914 19595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 14:55:30 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:30.715973 19595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "e3936e8d7e7544729a509ff7815acfa0"
format_stamp: "Formatted at 2026-06-20 14:55:30 on dist-test-slave-sms9"
server_key: "70c3d35cb79f0aa7722bea5a30efb11b"
server_key_iv: "3a47235c8bd3048e1f40e4253ed887a3"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:30.716732 19595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "e3936e8d7e7544729a509ff7815acfa0"
format_stamp: "Formatted at 2026-06-20 14:55:30 on dist-test-slave-sms9"
server_key: "70c3d35cb79f0aa7722bea5a30efb11b"
server_key_iv: "3a47235c8bd3048e1f40e4253ed887a3"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:30.722421 19595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 14:55:30.762778 19612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:30.764855 19595 fs_manager.cc:725] Time spent opening block manager: real 0.026s user 0.002s sys 0.000s
I20260620 14:55:30.765045 19595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "e3936e8d7e7544729a509ff7815acfa0"
format_stamp: "Formatted at 2026-06-20 14:55:30 on dist-test-slave-sms9"
server_key: "70c3d35cb79f0aa7722bea5a30efb11b"
server_key_iv: "3a47235c8bd3048e1f40e4253ed887a3"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:30.765208 19595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:30.783088 19595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:30.787196 19595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:30.787467 19595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:30.788224 19595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:30.789340 19595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:30.789494 19595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:30.789637 19595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:30.789729 19595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:30.800642 19595 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:37819
I20260620 14:55:30.801904 19595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:55:30.802987 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 19595
I20260620 14:55:30.803154 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:55:30.803447 7267 external_mini_cluster.cc:1428] Setting key 5ae9f9769db5208d5801c0701ac59b31
I20260620 14:55:30.811378 19727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:37819 every 8 connection(s)
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
Jun 20 14:55:30 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:30.835584 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:30.817232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:35483 (local address 127.7.24.193:40069)
0620 14:55:30.819614 (+ 2382us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:30.819659 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:30.820634 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:30.821213 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:30.821223 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:30.821520 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:30.822601 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:30.822626 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:30.824024 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:30.824029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:30.825305 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:30.825315 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:30.826096 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:30.826103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:30.826344 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:30.827184 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:30.827213 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:30.829101 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:30.831901 (+ 2800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:30.831909 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:30.831931 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:30.832341 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:30.832740 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:30.832743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:30.832745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:30.832965 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:30.833574 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:30.833582 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:30.833954 (+ 372us) client_negotiation.cc:769] Sending connection context
0620 14:55:30.834176 (+ 222us) client_negotiation.cc:240] Negotiation successful
0620 14:55:30.834583 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2171,"thread_start_us":182,"threads_started":1}
I20260620 14:55:30.836771 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:30.818632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:40069 (local address 127.7.24.254:35483)
0620 14:55:30.819371 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:30.819379 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:30.819780 (+ 401us) server_negotiation.cc:407] Connection header received
0620 14:55:30.820871 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:30.820875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:30.820941 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:30.821106 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:30.822801 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:30.823806 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:30.825446 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:30.825895 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:30.826799 (+ 904us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:30.829334 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:30.829341 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:30.829348 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:30.829387 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:30.831740 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:30.832508 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:30.832512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:30.832515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:30.832618 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:30.833109 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:30.833115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:30.833118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:30.833430 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:30.836360 (+ 2930us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:30.836475 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 14:55:30.836628 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":95,"threads_started":1}
I20260620 14:55:30.837939 19728 heartbeater.cc:347] Connected to a master server at 127.7.24.254:35483
I20260620 14:55:30.838228 19728 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:30.838933 19728 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:55:30.840828 19514 ts_manager.cc:194] Registered new tserver with Master: e3936e8d7e7544729a509ff7815acfa0 (127.7.24.193:37819)
I20260620 14:55:30.841915 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:48583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35483
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:45767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:55:30.842941 19514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:40069
Jun 20 14:55:30 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781967330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 14:55:30.981691 19734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:30.982146 19734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:30.982245 19734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:30.987596 19734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:30.987700 19734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:30.987815 19734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:55:30.994338 19734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:45767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.24.212:48583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:30.995527 19734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:30.996572 19734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:31.006701 19743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:31.006701 19741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:31.007424 19740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:31.007536 19734 server_base.cc:1034] running on GCE node
I20260620 14:55:31.007983 19734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:31.008773 19734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:31.010010 19734 hybrid_clock.cc:648] HybridClock initialized: now 1781967331009976 us; error 55 us; skew 500 ppm
Jun 20 14:55:31 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:31.013291 19734 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:55:31.014590 19734 webserver.cc:460] Webserver started at http://127.7.24.194:45971/ using document root <none> and password file <none>
I20260620 14:55:31.015434 19734 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:31.015538 19734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:31.015800 19734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 14:55:31 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:31.035408 19734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "6b80dab70c3b4b0e988e811902b648a7"
format_stamp: "Formatted at 2026-06-20 14:55:31 on dist-test-slave-sms9"
server_key: "b781d6d0b464aeb06e3ae04bda1cbbba"
server_key_iv: "71da0d4337bfd879abf16f2500340c17"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:31.036391 19734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "6b80dab70c3b4b0e988e811902b648a7"
format_stamp: "Formatted at 2026-06-20 14:55:31 on dist-test-slave-sms9"
server_key: "b781d6d0b464aeb06e3ae04bda1cbbba"
server_key_iv: "71da0d4337bfd879abf16f2500340c17"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:31.041952 19734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260620 14:55:31.057457 19750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:31.059294 19734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:55:31.059442 19734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "6b80dab70c3b4b0e988e811902b648a7"
format_stamp: "Formatted at 2026-06-20 14:55:31 on dist-test-slave-sms9"
server_key: "b781d6d0b464aeb06e3ae04bda1cbbba"
server_key_iv: "71da0d4337bfd879abf16f2500340c17"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:31.059654 19734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:31.078276 19734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:31.082175 19734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:31.082420 19734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:31.083405 19734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:31.084450 19734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:31.084555 19734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:31.084646 19734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:31.084723 19734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:31.095883 19734 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:41125
I20260620 14:55:31.096110 19863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:41125 every 8 connection(s)
I20260620 14:55:31.097076 19734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:55:31.097477 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 19734
I20260620 14:55:31.097612 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:55:31.097971 7267 external_mini_cluster.cc:1428] Setting key 9dabfcfa9e4e849a4410ca61f0369190
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
Jun 20 14:55:31 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:31.134290 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:31.101852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:33719 (local address 127.7.24.254:35483)
0620 14:55:31.101976 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:31.101981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:31.107103 (+ 5122us) server_negotiation.cc:407] Connection header received
0620 14:55:31.108436 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:31.108440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:31.108505 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:31.108570 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:31.110395 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:31.111258 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:31.112986 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:31.113344 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:31.113433 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:31.128764 (+ 15331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:31.128786 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:31.128790 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:31.128826 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:31.130833 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:31.132021 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:31.132026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:31.132028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:31.132122 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:31.132746 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:31.132750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:31.132752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:31.133027 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:31.133104 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:31.133949 (+ 845us) server_negotiation.cc:299] Negotiation successful
0620 14:55:31.134092 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 14:55:31.136689 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:31.101043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:35483 (local address 127.7.24.194:33719)
0620 14:55:31.106879 (+ 5836us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:31.106958 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:31.108146 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:31.109051 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:31.109062 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:31.109457 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:31.110197 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:31.110219 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:31.111427 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:31.111430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:31.112822 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:31.112846 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:31.113660 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:31.113665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:31.113896 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:31.125607 (+ 11711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:31.125639 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:31.128385 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:31.131373 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:31.131380 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:31.131401 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:31.131795 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:31.132287 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:31.132291 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:31.132293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:31.132625 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:31.133249 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:31.133256 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:31.133618 (+ 362us) client_negotiation.cc:769] Sending connection context
0620 14:55:31.135019 (+ 1401us) client_negotiation.cc:240] Negotiation successful
0620 14:55:31.135436 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5539,"thread_start_us":149,"threads_started":1}
I20260620 14:55:31.138557 19864 heartbeater.cc:347] Connected to a master server at 127.7.24.254:35483
I20260620 14:55:31.138805 19864 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:31.139400 19864 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
I20260620 14:55:31.140758 19514 ts_manager.cc:194] Registered new tserver with Master: 6b80dab70c3b4b0e988e811902b648a7 (127.7.24.194:41125)
I20260620 14:55:31.141537 19514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:33719
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:55:31.145002 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:48583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35483
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:45767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:31.307225 19869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:31.307724 19869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:31.307868 19869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:31.314814 19869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:31.314985 19869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:31.315148 19869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:55:31.324326 19869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:45767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.24.212:48583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:31.326149 19869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:31.327749 19869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:31.346053 19874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:31.346180 19869 server_base.cc:1034] running on GCE node
W20260620 14:55:31.346053 19875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:31.346640 19877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:31.347287 19869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:31.348210 19869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:31.349406 19869 hybrid_clock.cc:648] HybridClock initialized: now 1781967331349325 us; error 96 us; skew 500 ppm
Jun 20 14:55:31 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:31.353483 19869 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:55:31.354786 19869 webserver.cc:460] Webserver started at http://127.7.24.195:33857/ using document root <none> and password file <none>
I20260620 14:55:31.355620 19869 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:31.355707 19869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:31.355964 19869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 14:55:31 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:31.373327 19869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "d408881677de4762a5762c64d5ad18c4"
format_stamp: "Formatted at 2026-06-20 14:55:31 on dist-test-slave-sms9"
server_key: "8928bbda0ead5edb2bc1d3efefb8bf91"
server_key_iv: "3aff752e7d6c9d54300af0beec6521ad"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:31.374185 19869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "d408881677de4762a5762c64d5ad18c4"
format_stamp: "Formatted at 2026-06-20 14:55:31 on dist-test-slave-sms9"
server_key: "8928bbda0ead5edb2bc1d3efefb8bf91"
server_key_iv: "3aff752e7d6c9d54300af0beec6521ad"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:31.386082 19869 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.000s sys 0.003s
I20260620 14:55:31.405886 19884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:31.407783 19869 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.002s
I20260620 14:55:31.407938 19869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "d408881677de4762a5762c64d5ad18c4"
format_stamp: "Formatted at 2026-06-20 14:55:31 on dist-test-slave-sms9"
server_key: "8928bbda0ead5edb2bc1d3efefb8bf91"
server_key_iv: "3aff752e7d6c9d54300af0beec6521ad"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:31.408128 19869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Jun 20 14:55:31 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:31.433705 19869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:31.437294 19869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:31.437477 19869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:31.438124 19869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:31.439177 19869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:31.439236 19869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:31.439291 19869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:31.439319 19869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:31.448849 19869 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:36723
I20260620 14:55:31.449919 19869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:55:31.453372 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 19869
I20260620 14:55:31.453510 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:55:31.453835 7267 external_mini_cluster.cc:1428] Setting key a30291f0248774f101ebf9c5c59295bb
I20260620 14:55:31.464277 19997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:36723 every 8 connection(s)
Jun 20 14:55:31 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:31.493252 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:31.465310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:58159 (local address 127.7.24.254:35483)
0620 14:55:31.465466 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:31.465471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:31.471234 (+ 5763us) server_negotiation.cc:407] Connection header received
0620 14:55:31.472486 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:31.472490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:31.472545 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:31.472597 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:31.474232 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:31.475024 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:31.476635 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:31.476971 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:31.477053 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:31.488678 (+ 11625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:31.488683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:31.488686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:31.488720 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:31.490609 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:31.491285 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:31.491289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:31.491291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:31.491375 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:31.491827 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:31.491831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:31.491833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:31.492092 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:31.492172 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:31.492956 (+ 784us) server_negotiation.cc:299] Negotiation successful
0620 14:55:31.493087 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 14:55:31.495262 20000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:31.465201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:35483 (local address 127.7.24.195:58159)
0620 14:55:31.471065 (+ 5864us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:31.471129 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:31.472238 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:31.473023 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:31.473032 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:31.473395 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:31.474062 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:31.474082 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:31.475177 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:31.475180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:31.476501 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:31.476512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:31.477264 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:31.477268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:31.477532 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:31.482841 (+ 5309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:31.482865 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:31.488414 (+ 5549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:31.490767 (+ 2353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:31.490774 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:31.490794 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:31.491166 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:31.491491 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:31.491494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:31.491496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:31.491702 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:31.492263 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:31.492270 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:31.492639 (+ 369us) client_negotiation.cc:769] Sending connection context
0620 14:55:31.493725 (+ 1086us) client_negotiation.cc:240] Negotiation successful
0620 14:55:31.494107 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5592,"thread_start_us":159,"threads_started":1}
I20260620 14:55:31.496932 19998 heartbeater.cc:347] Connected to a master server at 127.7.24.254:35483
I20260620 14:55:31.497176 19998 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:31.497772 19998 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
I20260620 14:55:31.498948 19514 ts_manager.cc:194] Registered new tserver with Master: d408881677de4762a5762c64d5ad18c4 (127.7.24.195:36723)
I20260620 14:55:31.499634 19514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:58159
I20260620 14:55:31.502938 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:55:31.503751 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 19595
I20260620 14:55:31.513518 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 19734
I20260620 14:55:31.520829 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 19869
I20260620 14:55:31.525939 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 19482
I20260620 14:55:31.532908 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:35483
--webserver_interface=127.7.24.254
--webserver_port=33141
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:45767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:35483
--ranger_config_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:48583/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.24.212:48583/kms with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:31.641096 20001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:31.641389 20001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:31.641461 20001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:31.645511 20001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:55:31.645599 20001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:31.645638 20001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:31.645699 20001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:55:31.645738 20001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:55:31.651193 20001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:45767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.24.212:48583/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:35483
--ranger_config_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:35483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=33141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:31.652222 20001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:31.653210 20001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:31.661726 20009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:31.661782 20006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:31.661772 20007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:31.662017 20001 server_base.cc:1034] running on GCE node
I20260620 14:55:31.662674 20001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:31.663801 20001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:31.664999 20001 hybrid_clock.cc:648] HybridClock initialized: now 1781967331664967 us; error 52 us; skew 500 ppm
Jun 20 14:55:31 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:31.667797 20001 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:55:31.668615 20001 webserver.cc:460] Webserver started at http://127.7.24.254:33141/ using document root <none> and password file <none>
I20260620 14:55:31.669168 20001 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:31.669235 20001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 14:55:31 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:31.756023 20001 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 14:55:31.758285 20017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:31.759687 20001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:55:31.759820 20001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "dcf929f6d9f84a3ca7081e42d42e4014"
format_stamp: "Formatted at 2026-06-20 14:55:29 on dist-test-slave-sms9"
server_key: "eafdaa750fd7313a179e7873e6e97411"
server_key_iv: "2738a86b4993d0b1c90c21b379e8b846"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:31.760394 20001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:31.792888 20001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:31.795852 20001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:31.796056 20001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:31.803519 20001 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:35483
I20260620 14:55:31.803530 20069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:35483 every 8 connection(s)
I20260620 14:55:31.804606 20001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:55:31.808831 20070 sys_catalog.cc:263] Verifying existing consensus state
I20260620 14:55:31.810518 20070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014: Bootstrap starting.
I20260620 14:55:31.811069 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 20001
I20260620 14:55:31.811592 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:48583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:37819
--local_ip_for_outbound_sockets=127.7.24.193
--tserver_master_addrs=127.7.24.254:35483
--webserver_port=36831
--webserver_interface=127.7.24.193
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:45767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.24.212:48583/kms with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:55:31.828042 20070 log.cc:824] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:31.835556 20070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:55:31.836040 20070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014: Bootstrap complete.
I20260620 14:55:31.840550 20070 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } }
I20260620 14:55:31.841008 20070 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf929f6d9f84a3ca7081e42d42e4014, State: Initialized, Role: FOLLOWER
I20260620 14:55:31.841578 20070 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } }
I20260620 14:55:31.841698 20070 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:55:31.841790 20070 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:55:31.841917 20070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 1 FOLLOWER]: Advancing to term 2
I20260620 14:55:31.844058 20070 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } }
I20260620 14:55:31.844415 20070 leader_election.cc:302] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcf929f6d9f84a3ca7081e42d42e4014; no voters:
I20260620 14:55:31.844789 20070 leader_election.cc:288] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 14:55:31.844892 20074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 14:55:31.845198 20074 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [term 2 LEADER]: Becoming Leader. State: Replica: dcf929f6d9f84a3ca7081e42d42e4014, State: Running, Role: LEADER
I20260620 14:55:31.845507 20074 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } }
I20260620 14:55:31.846102 20070 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:55:31.846580 20076 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcf929f6d9f84a3ca7081e42d42e4014. Latest consensus state: current_term: 2 leader_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } } }
I20260620 14:55:31.846699 20076 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:31.846967 20075 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf929f6d9f84a3ca7081e42d42e4014" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 35483 } } }
I20260620 14:55:31.847066 20075 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:31.847167 20078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:55:31.850652 20078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:55:31.851190 20070 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 14:55:31.851398 20078 catalog_manager.cc:1259] Loaded cluster ID: 6267f642bf04489180f10b1c0a854eff
I20260620 14:55:31.851455 20078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:55:31.856709 20078 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:55:31.857630 20078 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014: Loaded TSK: 0
I20260620 14:55:31.858991 20078 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 14:55:31.929275 20072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:31.929642 20072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:31.929720 20072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:31.935885 20072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:31.935974 20072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:31.936070 20072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:55:31.944952 20072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:45767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.24.212:48583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:37819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=36831
--tserver_master_addrs=127.7.24.254:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:31.946506 20072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:31.947875 20072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:31.960250 20107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:31.961723 20072 server_base.cc:1034] running on GCE node
W20260620 14:55:31.965271 20110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:31.965335 20108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:31.966089 20072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:31.966931 20072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:31.968118 20072 hybrid_clock.cc:648] HybridClock initialized: now 1781967331968091 us; error 47 us; skew 500 ppm
Jun 20 14:55:31 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:31.971299 20072 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:55:31.972389 20072 webserver.cc:460] Webserver started at http://127.7.24.193:36831/ using document root <none> and password file <none>
I20260620 14:55:31.972995 20072 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:31.973104 20072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 14:55:31 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:32.006698 20072 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 14:55:32.009853 20118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:32.011551 20072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:55:32.011708 20072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "e3936e8d7e7544729a509ff7815acfa0"
format_stamp: "Formatted at 2026-06-20 14:55:30 on dist-test-slave-sms9"
server_key: "70c3d35cb79f0aa7722bea5a30efb11b"
server_key_iv: "3a47235c8bd3048e1f40e4253ed887a3"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:32.012281 20072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:32.036099 20072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:32.039927 20072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:32.040153 20072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:32.040800 20072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:32.041741 20072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:32.041796 20072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:32.041849 20072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:32.041873 20072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:32.052937 20072 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:37819
I20260620 14:55:32.053145 20231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:37819 every 8 connection(s)
I20260620 14:55:32.054150 20072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:55:32.060521 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 20072
I20260620 14:55:32.061868 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:48583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:41125
--local_ip_for_outbound_sockets=127.7.24.194
--tserver_master_addrs=127.7.24.254:35483
--webserver_port=45971
--webserver_interface=127.7.24.194
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:45767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.24.212:48583/kms with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 14:55:32 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:32.075451 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:32.056509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:35483 (local address 127.7.24.193:58011)
0620 14:55:32.057215 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:32.057267 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:32.058060 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:32.060714 (+ 2654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:32.060723 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:32.061070 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:32.061739 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:32.061761 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.063822 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.063826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:32.065315 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:32.065326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.067262 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.067266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:32.067456 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:32.068111 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:32.068128 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:32.069495 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:32.072024 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:32.072030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:32.072045 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:32.072349 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:32.072727 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:32.072730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:32.072732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:32.072863 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:32.073660 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:32.073665 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:32.073876 (+ 211us) client_negotiation.cc:769] Sending connection context
0620 14:55:32.074075 (+ 199us) client_negotiation.cc:240] Negotiation successful
0620 14:55:32.074411 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":125,"threads_started":1}
I20260620 14:55:32.075439 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:32.056368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:58011 (local address 127.7.24.254:35483)
0620 14:55:32.056943 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:32.056965 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:32.058364 (+ 1399us) server_negotiation.cc:407] Connection header received
0620 14:55:32.058675 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:32.058685 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:32.058959 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:32.059265 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:32.061948 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.063629 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.065463 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.066849 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.067027 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:32.069699 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:32.069705 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:32.069717 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:32.069745 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:32.071863 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:32.072488 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:32.072499 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:32.072504 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:32.072617 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:32.073023 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:32.073026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:32.073027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:32.073465 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:32.073600 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:32.074179 (+ 579us) server_negotiation.cc:299] Negotiation successful
0620 14:55:32.074415 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":100,"threads_started":1}
I20260620 14:55:32.078266 20232 heartbeater.cc:347] Connected to a master server at 127.7.24.254:35483
I20260620 14:55:32.078531 20232 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:32.079099 20232 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
W20260620 14:55:32.081828 20232 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:35483 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 14:55:32.081962 20232 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:32.082185 20232 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
I20260620 14:55:32.083344 20232 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:32.083536 20232 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
W20260620 14:55:32.084465 20232 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 14:55:32.230363 20236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:32.230737 20236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:32.230818 20236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:32.235437 20236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:32.235522 20236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:32.235625 20236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:55:32.241693 20236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:45767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.24.212:48583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:41125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=45971
--tserver_master_addrs=127.7.24.254:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:32.242945 20236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:32.244098 20236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:32.255354 20241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:32.255280 20242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:32.255576 20244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:32.255755 20236 server_base.cc:1034] running on GCE node
I20260620 14:55:32.256158 20236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:32.257099 20236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:32.258338 20236 hybrid_clock.cc:648] HybridClock initialized: now 1781967332258311 us; error 42 us; skew 500 ppm
Jun 20 14:55:32 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967332, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:32.264520 20236 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:55:32.265707 20236 webserver.cc:460] Webserver started at http://127.7.24.194:45971/ using document root <none> and password file <none>
I20260620 14:55:32.266309 20236 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:32.266427 20236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 14:55:32 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967332, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:32.287016 20236 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260620 14:55:32.289618 20252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:32.291018 20236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:55:32.291117 20236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "6b80dab70c3b4b0e988e811902b648a7"
format_stamp: "Formatted at 2026-06-20 14:55:31 on dist-test-slave-sms9"
server_key: "b781d6d0b464aeb06e3ae04bda1cbbba"
server_key_iv: "71da0d4337bfd879abf16f2500340c17"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:32.291697 20236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:32.303383 20236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:32.307056 20236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:32.307279 20236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:32.307998 20236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:32.308918 20236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:32.308971 20236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:32.309051 20236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:32.309087 20236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:32.319880 20236 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:41125
I20260620 14:55:32.321143 20236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:55:32.322710 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 20236
I20260620 14:55:32.323143 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.24.212:48583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:36723
--local_ip_for_outbound_sockets=127.7.24.195
--tserver_master_addrs=127.7.24.254:35483
--webserver_port=33857
--webserver_interface=127.7.24.195
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:45767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.24.212:48583/kms with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:55:32.325874 20365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:41125 every 8 connection(s)
Jun 20 14:55:32 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967332, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:32.343173 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:32.327679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:33359 (local address 127.7.24.254:35483)
0620 14:55:32.327843 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:32.327849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:32.329110 (+ 1261us) server_negotiation.cc:407] Connection header received
0620 14:55:32.330146 (+ 1036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:32.330149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:32.330210 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:32.330293 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:32.331462 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.332042 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.333266 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.333507 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.333632 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:32.338433 (+ 4801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:32.338436 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:32.338441 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:32.338519 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:32.340452 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:32.341179 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:32.341183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:32.341185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:32.341279 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:32.341728 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:32.341732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:32.341734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:32.341991 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:32.342069 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:32.342844 (+ 775us) server_negotiation.cc:299] Negotiation successful
0620 14:55:32.342998 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 14:55:32.345147 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:32.328420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:35483 (local address 127.7.24.194:33359)
0620 14:55:32.328936 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:32.328993 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:32.329925 (+ 932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:32.330528 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:32.330535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:32.330810 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:32.331294 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:32.331307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.332187 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.332195 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:32.333132 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:32.333142 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.333671 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.333675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:32.333846 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:32.334553 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:32.334569 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:32.338208 (+ 3639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:32.340622 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:32.340629 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:32.340648 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:32.341043 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:32.341407 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:32.341410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:32.341412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:32.341604 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:32.342156 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:32.342163 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:32.342550 (+ 387us) client_negotiation.cc:769] Sending connection context
0620 14:55:32.343658 (+ 1108us) client_negotiation.cc:240] Negotiation successful
0620 14:55:32.344066 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":166,"threads_started":1}
I20260620 14:55:32.346663 20366 heartbeater.cc:347] Connected to a master server at 127.7.24.254:35483
I20260620 14:55:32.346918 20366 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:32.347538 20366 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
W20260620 14:55:32.349314 20366 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:35483 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 14:55:32.349460 20366 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:32.349649 20366 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
I20260620 14:55:32.350601 20366 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:32.350780 20366 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
W20260620 14:55:32.351548 20366 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 14:55:32.486915 20368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:32.487319 20368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:32.487403 20368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:32.493700 20368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:32.493782 20368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:32.493876 20368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:55:32.501982 20368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:45767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.24.212:48583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:36723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=33857
--tserver_master_addrs=127.7.24.254:35483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:32.503466 20368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:32.504875 20368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:55:32.518226 20368 server_base.cc:1034] running on GCE node
W20260620 14:55:32.518532 20375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:32.520476 20376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:32.521082 20378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:32.521642 20368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:32.522526 20368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:32.523736 20368 hybrid_clock.cc:648] HybridClock initialized: now 1781967332523712 us; error 39 us; skew 500 ppm
Jun 20 14:55:32 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967332, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:32.527464 20368 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:55:32.528599 20368 webserver.cc:460] Webserver started at http://127.7.24.195:33857/ using document root <none> and password file <none>
I20260620 14:55:32.529321 20368 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:32.529438 20368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 14:55:32 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967332, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:32.557744 20368 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260620 14:55:32.560616 20386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:32.562181 20368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 14:55:32.562342 20368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "d408881677de4762a5762c64d5ad18c4"
format_stamp: "Formatted at 2026-06-20 14:55:31 on dist-test-slave-sms9"
server_key: "8928bbda0ead5edb2bc1d3efefb8bf91"
server_key_iv: "3aff752e7d6c9d54300af0beec6521ad"
server_key_version: "kuduclusterkey@0"
I20260620 14:55:32.563025 20368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:32.580044 20368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:32.583372 20368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:32.583582 20368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:32.584311 20368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:32.585145 20368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:32.585191 20368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:32.585232 20368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:32.585285 20368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:32.596318 20368 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:36723
I20260620 14:55:32.596472 20499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:36723 every 8 connection(s)
I20260620 14:55:32.597545 20368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:55:32.604233 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 20368
Jun 20 14:55:32 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967332, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:32.616919 20504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:32.605679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.7.24.254:35483)
0620 14:55:32.605985 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:32.605991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:32.606010 (+ 19us) server_negotiation.cc:407] Connection header received
0620 14:55:32.606063 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:32.606067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:32.606123 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:32.606243 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:32.607211 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.608171 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.609221 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.609470 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.609579 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:32.611316 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:32.611319 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:32.611322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:32.611352 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:32.614635 (+ 3283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:32.615109 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:32.615117 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:32.615118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:32.615227 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:32.615832 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:32.615837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:32.615839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:32.616074 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:32.616321 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:32.616456 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 14:55:32.616593 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":90,"threads_started":1}
I20260620 14:55:32.619369 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:32.602750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:46621 (local address 127.7.24.254:35483)
0620 14:55:32.602899 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:32.602904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:32.606138 (+ 3234us) server_negotiation.cc:407] Connection header received
0620 14:55:32.607416 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:32.607420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:32.607476 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:32.607546 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:32.608780 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.609622 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.611654 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.611955 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.612013 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:32.614916 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:32.614920 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:32.614923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:32.614950 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:32.616843 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:32.617552 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:32.617555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:32.617556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:32.617631 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:32.618072 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:32.618075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:32.618076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:32.618273 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:32.618367 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:32.619082 (+ 715us) server_negotiation.cc:299] Negotiation successful
0620 14:55:32.619202 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 14:55:32.621011 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:32.601565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:35483 (local address 127.7.24.195:46621)
0620 14:55:32.605990 (+ 4425us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:32.606038 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:32.607210 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:32.607730 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:32.607738 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:32.607999 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:32.608613 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:32.608630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.609757 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.609762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:32.611174 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:32.611185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:32.612539 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:32.612546 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:32.612790 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:32.613324 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:32.613340 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:32.614738 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:32.616984 (+ 2246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:32.616991 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:32.617007 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:32.617447 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:32.617733 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:32.617737 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:32.617739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:32.617938 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:32.618393 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:32.618400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:32.618780 (+ 380us) client_negotiation.cc:769] Sending connection context
0620 14:55:32.619572 (+ 792us) client_negotiation.cc:240] Negotiation successful
0620 14:55:32.619923 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":176,"threads_started":1}
I20260620 14:55:32.622594 20500 heartbeater.cc:347] Connected to a master server at 127.7.24.254:35483
I20260620 14:55:32.622851 20500 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:32.623394 20500 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
W20260620 14:55:32.624990 20500 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:35483 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 14:55:32.625133 20500 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:32.625344 20500 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
I20260620 14:55:32.626211 20500 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:32.626412 20500 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
W20260620 14:55:32.627243 20500 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 14:55:33 dist-test-slave-sms9 krb5kdc[19038](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781967333, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:33.084699 20232 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:33.085024 20232 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
I20260620 14:55:33.086200 20034 ts_manager.cc:194] Registered new tserver with Master: e3936e8d7e7544729a509ff7815acfa0 (127.7.24.193:37819)
I20260620 14:55:33.087900 20034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:58011
I20260620 14:55:33.351800 20366 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:33.352173 20366 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
I20260620 14:55:33.353039 20034 ts_manager.cc:194] Registered new tserver with Master: 6b80dab70c3b4b0e988e811902b648a7 (127.7.24.194:41125)
I20260620 14:55:33.353853 20034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:33359
Jun 20 14:55:33 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967333, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:33.627517 20500 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:33.627840 20500 heartbeater.cc:511] Master 127.7.24.254:35483 requested a full tablet report, sending...
I20260620 14:55:33.628702 20034 ts_manager.cc:194] Registered new tserver with Master: d408881677de4762a5762c64d5ad18c4 (127.7.24.195:36723)
I20260620 14:55:33.629527 20034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:46621
I20260620 14:55:33.632521 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 14:55:33 dist-test-slave-sms9 krb5kdc[19038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967333, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for HTTP/127.7.24.212@KRBTEST.COM
I20260620 14:55:33.648085 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:33.634207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.7.24.254:35483)
0620 14:55:33.635593 (+ 1386us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:33.635598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:33.635835 (+ 237us) server_negotiation.cc:407] Connection header received
0620 14:55:33.636026 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:33.636031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:33.636088 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:33.636151 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:33.637156 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.637946 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.639093 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.639483 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.639573 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:33.641456 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:33.641460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:33.641464 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:33.641490 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:33.643197 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:33.643774 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:33.643777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:33.643779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:33.643864 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:33.644139 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:33.644142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:33.644144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:33.644708 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:33.644830 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:33.647767 (+ 2937us) server_negotiation.cc:299] Negotiation successful
0620 14:55:33.647894 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1295,"thread_start_us":79,"threads_started":1}
I20260620 14:55:33.654078 20034 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:55:33.659210 20034 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:55:33.710537 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:33.697564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:36723 (local address 127.0.0.1:57964)
0620 14:55:33.700965 (+ 3401us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:33.701022 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:33.701150 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:33.702781 (+ 1631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:33.702788 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:33.702813 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:33.702963 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:33.702970 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.705568 (+ 2598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.705571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:33.708672 (+ 3101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:33.708685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.709898 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.709908 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:33.710051 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:33.710096 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 14:55:33.710217 (+ 121us) client_negotiation.cc:240] Negotiation successful
0620 14:55:33.710341 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3205,"thread_start_us":112,"threads_started":1}
I20260620 14:55:33.711321 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:33.697696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57964 (local address 127.7.24.195:36723)
0620 14:55:33.701484 (+ 3788us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:33.701491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:33.701516 (+ 25us) server_negotiation.cc:407] Connection header received
0620 14:55:33.701594 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:33.701600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:33.702602 (+ 1002us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:33.703122 (+ 520us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:33.703918 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.705107 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.708834 (+ 3727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.709754 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.709871 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:33.710044 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:33.710943 (+ 899us) server_negotiation.cc:299] Negotiation successful
0620 14:55:33.711064 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3653,"thread_start_us":90,"threads_started":1}
I20260620 14:55:33.714866 20434 tablet_service.cc:1467] Processing CreateTablet for tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b (DEFAULT_TABLE table=test-table [id=43ede3fa687648c8874972ced4138bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:55:33.717708 20434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:33.731981 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:33.698709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:41125 (local address 127.0.0.1:33212)
0620 14:55:33.705332 (+ 6623us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:33.705347 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:33.705446 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:33.707272 (+ 1826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:33.707275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:33.707289 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:33.710181 (+ 2892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:33.710188 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.728745 (+ 18557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.728749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:33.729986 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:33.730001 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.731464 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.731467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:33.731617 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:33.731647 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 14:55:33.731726 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 14:55:33.731815 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6553,"mutex_wait_us":2766,"thread_start_us":83,"threads_started":1}
I20260620 14:55:33.733069 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:33.699092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33212 (local address 127.7.24.194:41125)
0620 14:55:33.705896 (+ 6804us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:33.705915 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:33.705939 (+ 24us) server_negotiation.cc:407] Connection header received
0620 14:55:33.706009 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:33.706013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:33.706207 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:33.706347 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:33.710334 (+ 3987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.728589 (+ 18255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.730159 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.731124 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.731232 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:33.731385 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:33.732815 (+ 1430us) server_negotiation.cc:299] Negotiation successful
0620 14:55:33.732933 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6702,"thread_start_us":122,"threads_started":1}
I20260620 14:55:33.737001 20282 tablet_service.cc:1467] Processing CreateTablet for tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b (DEFAULT_TABLE table=test-table [id=43ede3fa687648c8874972ced4138bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:55:33.738157 20282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:33.741385 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:33.699747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:37819 (local address 127.0.0.1:55318)
0620 14:55:33.705577 (+ 5830us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:33.705593 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:33.707177 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:33.734187 (+ 27010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:33.734190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:33.734213 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:33.734310 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:33.734316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.738629 (+ 4313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.738632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:33.739796 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:33.739808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.740980 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.740984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:33.741075 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:33.741105 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 14:55:33.741157 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 14:55:33.741235 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5765,"thread_start_us":83,"threads_started":1}
I20260620 14:55:33.742311 20534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:33.700143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55318 (local address 127.7.24.193:37819)
0620 14:55:33.733486 (+ 33343us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:33.733494 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:33.733523 (+ 29us) server_negotiation.cc:407] Connection header received
0620 14:55:33.733760 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:33.733768 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:33.734025 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:33.734493 (+ 468us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:33.734641 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.735731 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.739965 (+ 4234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.740767 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.741869 (+ 1102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:33.742021 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:33.742073 (+ 52us) server_negotiation.cc:299] Negotiation successful
0620 14:55:33.742171 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33205,"thread_start_us":68,"threads_started":1}
I20260620 14:55:33.745426 20166 tablet_service.cc:1467] Processing CreateTablet for tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b (DEFAULT_TABLE table=test-table [id=43ede3fa687648c8874972ced4138bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:55:33.746529 20166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:33.750801 20535 tablet_bootstrap.cc:492] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: Bootstrap starting.
I20260620 14:55:33.753320 20535 tablet_bootstrap.cc:654] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:33.754309 20535 log.cc:824] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:33.755141 20536 tablet_bootstrap.cc:492] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: Bootstrap starting.
I20260620 14:55:33.757403 20536 tablet_bootstrap.cc:654] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:33.758265 20536 log.cc:824] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:33.762261 20535 tablet_bootstrap.cc:492] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: No bootstrap required, opened a new log
I20260620 14:55:33.762506 20535 ts_tablet_manager.cc:1397] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.002s
I20260620 14:55:33.764420 20538 tablet_bootstrap.cc:492] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0: Bootstrap starting.
I20260620 14:55:33.766870 20535 raft_consensus.cc:348] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d408881677de4762a5762c64d5ad18c4" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36723 } } peers { permanent_uuid: "6b80dab70c3b4b0e988e811902b648a7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 41125 } } peers { permanent_uuid: "e3936e8d7e7544729a509ff7815acfa0" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37819 } }
I20260620 14:55:33.767207 20535 raft_consensus.cc:374] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:33.767270 20535 raft_consensus.cc:729] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d408881677de4762a5762c64d5ad18c4, State: Initialized, Role: FOLLOWER
I20260620 14:55:33.767760 20535 consensus_queue.cc:260] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d408881677de4762a5762c64d5ad18c4" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36723 } } peers { permanent_uuid: "6b80dab70c3b4b0e988e811902b648a7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 41125 } } peers { permanent_uuid: "e3936e8d7e7544729a509ff7815acfa0" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37819 } }
I20260620 14:55:33.768916 20535 ts_tablet_manager.cc:1428] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260620 14:55:33.769289 20500 heartbeater.cc:503] Master 127.7.24.254:35483 was elected leader, sending a full tablet report...
I20260620 14:55:33.771184 20536 tablet_bootstrap.cc:492] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: No bootstrap required, opened a new log
I20260620 14:55:33.771456 20536 ts_tablet_manager.cc:1397] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20260620 14:55:33.775784 20536 raft_consensus.cc:348] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d408881677de4762a5762c64d5ad18c4" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36723 } } peers { permanent_uuid: "6b80dab70c3b4b0e988e811902b648a7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 41125 } } peers { permanent_uuid: "e3936e8d7e7544729a509ff7815acfa0" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37819 } }
I20260620 14:55:33.776299 20538 tablet_bootstrap.cc:654] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:33.783599 20538 log.cc:824] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:33.784242 20536 raft_consensus.cc:374] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:33.784379 20536 raft_consensus.cc:729] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b80dab70c3b4b0e988e811902b648a7, State: Initialized, Role: FOLLOWER
I20260620 14:55:33.785029 20536 consensus_queue.cc:260] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d408881677de4762a5762c64d5ad18c4" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36723 } } peers { permanent_uuid: "6b80dab70c3b4b0e988e811902b648a7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 41125 } } peers { permanent_uuid: "e3936e8d7e7544729a509ff7815acfa0" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37819 } }
I20260620 14:55:33.786331 20536 ts_tablet_manager.cc:1428] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260620 14:55:33.786798 20366 heartbeater.cc:503] Master 127.7.24.254:35483 was elected leader, sending a full tablet report...
I20260620 14:55:33.798789 20538 tablet_bootstrap.cc:492] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0: No bootstrap required, opened a new log
I20260620 14:55:33.799036 20538 ts_tablet_manager.cc:1397] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.002s
I20260620 14:55:33.802549 20538 raft_consensus.cc:348] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d408881677de4762a5762c64d5ad18c4" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36723 } } peers { permanent_uuid: "6b80dab70c3b4b0e988e811902b648a7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 41125 } } peers { permanent_uuid: "e3936e8d7e7544729a509ff7815acfa0" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37819 } }
I20260620 14:55:33.802920 20538 raft_consensus.cc:374] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:33.803017 20538 raft_consensus.cc:729] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3936e8d7e7544729a509ff7815acfa0, State: Initialized, Role: FOLLOWER
I20260620 14:55:33.803599 20538 consensus_queue.cc:260] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d408881677de4762a5762c64d5ad18c4" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36723 } } peers { permanent_uuid: "6b80dab70c3b4b0e988e811902b648a7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 41125 } } peers { permanent_uuid: "e3936e8d7e7544729a509ff7815acfa0" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37819 } }
I20260620 14:55:33.804714 20538 ts_tablet_manager.cc:1428] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260620 14:55:33.805104 20232 heartbeater.cc:503] Master 127.7.24.254:35483 was elected leader, sending a full tablet report...
W20260620 14:55:33.807180 20233 tablet.cc:2369] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 14:55:33.829084 20367 tablet.cc:2369] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 14:55:33.850332 20501 tablet.cc:2369] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:55:33.993551 20540 raft_consensus.cc:482] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:55:33.993803 20540 raft_consensus.cc:504] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d408881677de4762a5762c64d5ad18c4" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36723 } } peers { permanent_uuid: "6b80dab70c3b4b0e988e811902b648a7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 41125 } } peers { permanent_uuid: "e3936e8d7e7544729a509ff7815acfa0" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37819 } }
I20260620 14:55:33.995054 20540 leader_election.cc:288] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b80dab70c3b4b0e988e811902b648a7 (127.7.24.194:41125), e3936e8d7e7544729a509ff7815acfa0 (127.7.24.193:37819)
I20260620 14:55:33.999683 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:33.995411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:57359 (local address 127.7.24.194:41125)
0620 14:55:33.995555 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:33.995560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:33.995876 (+ 316us) server_negotiation.cc:407] Connection header received
0620 14:55:33.996112 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:33.996116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:33.996167 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:33.996248 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:33.996661 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.997418 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.998629 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.999119 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.999243 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:33.999295 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:33.999480 (+ 185us) server_negotiation.cc:299] Negotiation successful
0620 14:55:33.999560 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 14:55:34.000082 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:33.995270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:41125 (local address 127.7.24.195:57359)
0620 14:55:33.995781 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:33.995797 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:33.995958 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:33.996285 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:33.996288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:33.996305 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:33.996468 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:33.996474 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.997548 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.997551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:33.998429 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:33.998438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.999243 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.999246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:33.999335 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:33.999360 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 14:55:33.999856 (+ 496us) client_negotiation.cc:240] Negotiation successful
0620 14:55:33.999938 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":182,"threads_started":1}
I20260620 14:55:34.000097 20534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:33.995894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:48489 (local address 127.7.24.193:37819)
0620 14:55:33.996019 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:33.996023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:33.996212 (+ 189us) server_negotiation.cc:407] Connection header received
0620 14:55:33.996365 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:33.996369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:33.996420 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:33.996471 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:33.996960 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.997536 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.998953 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.999555 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.999669 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:33.999716 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:33.999916 (+ 200us) server_negotiation.cc:299] Negotiation successful
0620 14:55:33.999999 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 14:55:34.000486 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:33.995753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:37819 (local address 127.7.24.195:48489)
0620 14:55:33.996118 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:33.996130 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:33.996239 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:33.996672 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:33.996674 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:33.996691 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:33.996837 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:33.996843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.997668 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.997672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:33.998815 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:33.998826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:33.999662 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:33.999665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:33.999765 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:33.999800 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 14:55:34.000255 (+ 455us) client_negotiation.cc:240] Negotiation successful
0620 14:55:34.000346 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":101,"threads_started":1}
I20260620 14:55:34.000679 20302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1b13a6d1a394bdb9e48e0ecde6b7a9b" candidate_uuid: "d408881677de4762a5762c64d5ad18c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b80dab70c3b4b0e988e811902b648a7" is_pre_election: true
I20260620 14:55:34.000993 20302 raft_consensus.cc:2454] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d408881677de4762a5762c64d5ad18c4 in term 0.
I20260620 14:55:34.001070 20186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1b13a6d1a394bdb9e48e0ecde6b7a9b" candidate_uuid: "d408881677de4762a5762c64d5ad18c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3936e8d7e7544729a509ff7815acfa0" is_pre_election: true
I20260620 14:55:34.001317 20186 raft_consensus.cc:2454] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d408881677de4762a5762c64d5ad18c4 in term 0.
I20260620 14:55:34.001490 20389 leader_election.cc:302] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b80dab70c3b4b0e988e811902b648a7, d408881677de4762a5762c64d5ad18c4; no voters:
I20260620 14:55:34.001806 20540 raft_consensus.cc:2781] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:55:34.001896 20540 raft_consensus.cc:482] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:55:34.001942 20540 raft_consensus.cc:3037] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:34.003189 20540 raft_consensus.cc:504] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d408881677de4762a5762c64d5ad18c4" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36723 } } peers { permanent_uuid: "6b80dab70c3b4b0e988e811902b648a7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 41125 } } peers { permanent_uuid: "e3936e8d7e7544729a509ff7815acfa0" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37819 } }
I20260620 14:55:34.003520 20540 leader_election.cc:288] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [CANDIDATE]: Term 1 election: Requested vote from peers 6b80dab70c3b4b0e988e811902b648a7 (127.7.24.194:41125), e3936e8d7e7544729a509ff7815acfa0 (127.7.24.193:37819)
I20260620 14:55:34.003854 20302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1b13a6d1a394bdb9e48e0ecde6b7a9b" candidate_uuid: "d408881677de4762a5762c64d5ad18c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b80dab70c3b4b0e988e811902b648a7"
I20260620 14:55:34.003999 20186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1b13a6d1a394bdb9e48e0ecde6b7a9b" candidate_uuid: "d408881677de4762a5762c64d5ad18c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3936e8d7e7544729a509ff7815acfa0"
I20260620 14:55:34.004025 20302 raft_consensus.cc:3037] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:34.004148 20186 raft_consensus.cc:3037] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:34.005326 20302 raft_consensus.cc:2454] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d408881677de4762a5762c64d5ad18c4 in term 1.
I20260620 14:55:34.005475 20186 raft_consensus.cc:2454] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d408881677de4762a5762c64d5ad18c4 in term 1.
I20260620 14:55:34.005604 20389 leader_election.cc:302] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b80dab70c3b4b0e988e811902b648a7, d408881677de4762a5762c64d5ad18c4; no voters:
I20260620 14:55:34.005767 20540 raft_consensus.cc:2781] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:55:34.005975 20540 raft_consensus.cc:686] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 1 LEADER]: Becoming Leader. State: Replica: d408881677de4762a5762c64d5ad18c4, State: Running, Role: LEADER
I20260620 14:55:34.006297 20540 consensus_queue.cc:237] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d408881677de4762a5762c64d5ad18c4" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36723 } } peers { permanent_uuid: "6b80dab70c3b4b0e988e811902b648a7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 41125 } } peers { permanent_uuid: "e3936e8d7e7544729a509ff7815acfa0" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37819 } }
I20260620 14:55:34.009737 20034 catalog_manager.cc:5495] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d408881677de4762a5762c64d5ad18c4 (127.7.24.195). New cstate: current_term: 1 leader_uuid: "d408881677de4762a5762c64d5ad18c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d408881677de4762a5762c64d5ad18c4" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b80dab70c3b4b0e988e811902b648a7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 41125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3936e8d7e7544729a509ff7815acfa0" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37819 } health_report { overall_health: UNKNOWN } } }
I20260620 14:55:34.069255 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:34.065175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.7.24.195:36723)
0620 14:55:34.065317 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:34.065320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:34.065333 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:55:34.065473 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:34.065475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:34.065533 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:34.065620 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:55:34.065867 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:34.066530 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:34.067667 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:34.067924 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:34.068079 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:34.068354 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:55:34.068781 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:55:34.068872 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:34.069002 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 14:55:34.069052 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 14:55:34.075027 20302 raft_consensus.cc:1261] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7 [term 1 FOLLOWER]: Refusing update from remote peer d408881677de4762a5762c64d5ad18c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:55:34.075166 20186 raft_consensus.cc:1261] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P e3936e8d7e7544729a509ff7815acfa0 [term 1 FOLLOWER]: Refusing update from remote peer d408881677de4762a5762c64d5ad18c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:55:34.075815 20540 consensus_queue.cc:1035] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b80dab70c3b4b0e988e811902b648a7" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 41125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:55:34.076020 20553 consensus_queue.cc:1035] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3936e8d7e7544729a509ff7815acfa0" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 37819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:55:34.086195 20558 mvcc.cc:205] Tried to move back new op lower bound from 7298938200363986944 to 7298938200092549120. Current Snapshot: MvccSnapshot[applied={T|T < 7298938200363986944}]
I20260620 14:55:34.088697 20557 mvcc.cc:205] Tried to move back new op lower bound from 7298938200363986944 to 7298938200092549120. Current Snapshot: MvccSnapshot[applied={T|T < 7298938200363986944}]
I20260620 14:55:34.093603 20034 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:55:34.093780 20034 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:55:34.096282 20034 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dcf929f6d9f84a3ca7081e42d42e4014: Sending DeleteTablet for 3 replicas of tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b
I20260620 14:55:34.097060 20434 tablet_service.cc:1514] Processing DeleteTablet for tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:55:34 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:57964
I20260620 14:55:34.097595 20282 tablet_service.cc:1514] Processing DeleteTablet for tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:55:34 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:33212
I20260620 14:55:34.099040 20166 tablet_service.cc:1514] Processing DeleteTablet for tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:55:34 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:55318
I20260620 14:55:34.099669 20566 tablet_replica.cc:330] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: stopping tablet replica
I20260620 14:55:34.100005 20566 raft_consensus.cc:2229] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:55:34.100428 20566 raft_consensus.cc:2258] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:55:34.102119 20566 ts_tablet_manager.cc:1905] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:55:34.102327 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 20072
I20260620 14:55:34.103860 20567 tablet_replica.cc:330] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: stopping tablet replica
I20260620 14:55:34.104435 20567 raft_consensus.cc:2229] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:55:34.105085 20567 raft_consensus.cc:2258] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:55:34.105532 20566 ts_tablet_manager.cc:1918] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:55:34.105633 20566 log.cc:1196] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/a1b13a6d1a394bdb9e48e0ecde6b7a9b
I20260620 14:55:34.105998 20566 ts_tablet_manager.cc:1939] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P d408881677de4762a5762c64d5ad18c4: Deleting consensus metadata
I20260620 14:55:34.107265 20021 catalog_manager.cc:4841] TS d408881677de4762a5762c64d5ad18c4 (127.7.24.195:36723): tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b (table test-table [id=43ede3fa687648c8874972ced4138bcd]) successfully deleted
I20260620 14:55:34.107628 20567 ts_tablet_manager.cc:1905] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:55:34.109887 20567 ts_tablet_manager.cc:1918] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:55:34.109989 20567 log.cc:1196] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/a1b13a6d1a394bdb9e48e0ecde6b7a9b
I20260620 14:55:34.110268 20567 ts_tablet_manager.cc:1939] T a1b13a6d1a394bdb9e48e0ecde6b7a9b P 6b80dab70c3b4b0e988e811902b648a7: Deleting consensus metadata
I20260620 14:55:34.111564 20020 catalog_manager.cc:4841] TS 6b80dab70c3b4b0e988e811902b648a7 (127.7.24.194:41125): tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b (table test-table [id=43ede3fa687648c8874972ced4138bcd]) successfully deleted
W20260620 14:55:34.115186 20020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:37819 (error 108)
I20260620 14:55:34.115796 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 20236
I20260620 14:55:34.118279 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:34.117949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:37819 (local address 127.0.0.1:55332)
0620 14:55:34.118091 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:34.118174 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:37819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260620 14:55:34.118417 20020 catalog_manager.cc:4584] TS e3936e8d7e7544729a509ff7815acfa0 (127.7.24.193:37819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1b13a6d1a394bdb9e48e0ecde6b7a9b: Network error: Client connection negotiation failed: client connection to 127.7.24.193:37819: connect: Connection refused (error 111)
I20260620 14:55:34.122634 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 20368
I20260620 14:55:34.128953 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 20001
I20260620 14:55:34.136915 7267 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 14:55:34.471937 7267 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 14:55:34.472043 7267 mini_ranger.cc:66] Stopping Ranger...
I20260620 14:55:34.817365 7267 mini_ranger.cc:68] Stopped Ranger
2026-06-20 14:55:34.817 UTC [19068] LOG: received smart shutdown request
2026-06-20 14:55:34.821 UTC [19068] LOG: background worker "logical replication launcher" (PID 19077) exited with exit code 1
2026-06-20 14:55:34.821 UTC [19072] LOG: shutting down
2026-06-20 14:55:34.989 UTC [19068] LOG: database system is shut down
2026-06-20T14:55:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50093 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:55:35 dist-test-slave-sms9 krb5kdc[20578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:55:35 dist-test-slave-sms9 krb5kdc[20578](info): set up 2 sockets
Jun 20 14:55:35 dist-test-slave-sms9 krb5kdc[20578](info): commencing operation
krb5kdc: starting...
W20260620 14:55:36.162744 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:55:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:55:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:55:36.294680 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:42677
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:42677,127.7.24.253:35725,127.7.24.252:34779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:36.405391 20591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:36.405699 20591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:36.405784 20591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:36.409957 20591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:55:36.410046 20591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:36.410084 20591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:36.410135 20591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:55:36.410171 20591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:55:36.415591 20591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:42677,127.7.24.253:35725,127.7.24.252:34779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:42677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:36.416692 20591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:36.417727 20591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:55:36.426380 20591 server_base.cc:1034] running on GCE node
W20260620 14:55:36.426352 20599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:36.426318 20596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:36.426324 20597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:36.427225 20591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:36.428233 20591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:36.429442 20591 hybrid_clock.cc:648] HybridClock initialized: now 1781967336429435 us; error 30 us; skew 500 ppm
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:36.432310 20591 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:55:36.433223 20591 webserver.cc:460] Webserver started at http://127.7.24.254:33423/ using document root <none> and password file <none>
I20260620 14:55:36.433741 20591 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:36.433797 20591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:36.433961 20591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:55:36.436050 20591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "d7a49fe17cec4c5c8f24000b1939befd"
format_stamp: "Formatted at 2026-06-20 14:55:36 on dist-test-slave-sms9"
server_key: "e3dad82c575f12298d906fbb293b0c97"
server_key_iv: "148f739d75a0f220e6866de326dc79bd"
server_key_version: "encryptionkey@0"
I20260620 14:55:36.436571 20591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "d7a49fe17cec4c5c8f24000b1939befd"
format_stamp: "Formatted at 2026-06-20 14:55:36 on dist-test-slave-sms9"
server_key: "e3dad82c575f12298d906fbb293b0c97"
server_key_iv: "148f739d75a0f220e6866de326dc79bd"
server_key_version: "encryptionkey@0"
I20260620 14:55:36.440367 20591 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260620 14:55:36.442875 20606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:36.444191 20591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:55:36.444294 20591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "d7a49fe17cec4c5c8f24000b1939befd"
format_stamp: "Formatted at 2026-06-20 14:55:36 on dist-test-slave-sms9"
server_key: "e3dad82c575f12298d906fbb293b0c97"
server_key_iv: "148f739d75a0f220e6866de326dc79bd"
server_key_version: "encryptionkey@0"
I20260620 14:55:36.444391 20591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:36.459317 20591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:36.462386 20591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:36.462634 20591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:36.470101 20591 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:42677
I20260620 14:55:36.470115 20658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:42677 every 8 connection(s)
I20260620 14:55:36.471174 20591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:55:36.471385 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 20591
I20260620 14:55:36.471534 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:55:36.471885 7267 external_mini_cluster.cc:1428] Setting key c9f0f2067d753803a7ba4591031126bd
I20260620 14:55:36.474603 20659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:36.478523 20659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.24.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.253@KRBTEST.COM" created.
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:36.496212 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.481531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57014 (local address 127.7.24.254:42677)
0620 14:55:36.483481 (+ 1950us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.483489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.483509 (+ 20us) server_negotiation.cc:407] Connection header received
0620 14:55:36.483620 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.483624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.483797 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.483943 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.484904 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.485999 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.487123 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.487796 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.487984 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.490747 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.490756 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.490771 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.490802 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.492680 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.493142 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.493147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.493148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.493296 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.493709 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.493712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.493713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.494276 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.494511 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.495041 (+ 530us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.495355 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":1844,"spinlock_wait_cycles":54528,"thread_start_us":167,"threads_started":1}
I20260620 14:55:36.496210 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.481531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42677 (local address 127.0.0.1:57014)
0620 14:55:36.482155 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.482200 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:36.483132 (+ 932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:36.484025 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:36.484034 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:36.484483 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:36.484742 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.484755 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.486134 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.486137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.486994 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.487003 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.488009 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.488012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.488110 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.488748 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:36.488764 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:36.490583 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:36.492791 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.492798 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.492806 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.493031 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.493402 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.493405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.493406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.493522 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.494583 (+ 1061us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:36.494589 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:36.494763 (+ 174us) client_negotiation.cc:769] Sending connection context
0620 14:55:36.494922 (+ 159us) client_negotiation.cc:240] Negotiation successful
0620 14:55:36.495229 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":167,"threads_started":1}
I20260620 14:55:36.498782 20659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:55:36.499670 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.499321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35725 (local address 127.0.0.1:50682)
0620 14:55:36.499444 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.499585 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:35725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 14:55:36.499922 20610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.24.253:35725: connect: Connection refused (error 111)
I20260620 14:55:36.500816 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.500615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35725 (local address 127.0.0.1:50694)
0620 14:55:36.500721 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.500765 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:35725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 14:55:36.501049 20659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.253:35725: Network error: Client connection negotiation failed: client connection to 127.7.24.253:35725: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab.
Entry for principal kudu/127.7.24.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab.
I20260620 14:55:36.502074 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.253:35725
--webserver_interface=127.7.24.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab
--principal=kudu/127.7.24.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:42677,127.7.24.253:35725,127.7.24.252:34779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:55:36.550344 20659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } attempt: 1
I20260620 14:55:36.551333 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.550958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35725 (local address 127.0.0.1:50702)
0620 14:55:36.551150 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.551238 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:35725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260620 14:55:36.552109 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.551868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35725 (local address 127.0.0.1:50714)
0620 14:55:36.551985 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.552038 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:35725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 14:55:36.552256 20659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.253:35725: Network error: Client connection negotiation failed: client connection to 127.7.24.253:35725: connect: Connection refused (error 111)
W20260620 14:55:36.612376 20666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:36.612632 20666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:36.612711 20666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:36.616684 20666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:55:36.616743 20666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:36.616763 20666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:36.616832 20666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:55:36.616860 20666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:55:36.620450 20659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } attempt: 2
I20260620 14:55:36.621322 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.620968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35725 (local address 127.0.0.1:50720)
0620 14:55:36.621157 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.621231 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:35725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260620 14:55:36.622109 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.621888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35725 (local address 127.0.0.1:50734)
0620 14:55:36.621996 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.622047 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:35725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260620 14:55:36.622709 20659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.253:35725: Network error: Client connection negotiation failed: client connection to 127.7.24.253:35725: connect: Connection refused (error 111)
I20260620 14:55:36.622718 20666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:42677,127.7.24.253:35725,127.7.24.252:34779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab
--principal=kudu/127.7.24.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.253:35725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.24.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:36.623793 20666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:36.624872 20666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:36.633615 20674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:36.633626 20672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:36.633715 20671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:36.633785 20666 server_base.cc:1034] running on GCE node
I20260620 14:55:36.634572 20666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:36.635653 20666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:36.636873 20666 hybrid_clock.cc:648] HybridClock initialized: now 1781967336636833 us; error 43 us; skew 500 ppm
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:36.639787 20666 init.cc:377] Logged in from keytab as kudu/127.7.24.253@KRBTEST.COM (short username kudu)
I20260620 14:55:36.640724 20666 webserver.cc:460] Webserver started at http://127.7.24.253:46493/ using document root <none> and password file <none>
I20260620 14:55:36.641302 20666 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:36.641373 20666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:36.641613 20666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:55:36.643862 20666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/data/instance:
uuid: "056836de87d04c299cb9501c6ce64954"
format_stamp: "Formatted at 2026-06-20 14:55:36 on dist-test-slave-sms9"
server_key: "4edcc61e67962c9b432500dde0a03c6c"
server_key_iv: "c1f23cc23865d80d9f8cc9a0f9dff8d8"
server_key_version: "encryptionkey@0"
I20260620 14:55:36.644421 20666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/wal/instance:
uuid: "056836de87d04c299cb9501c6ce64954"
format_stamp: "Formatted at 2026-06-20 14:55:36 on dist-test-slave-sms9"
server_key: "4edcc61e67962c9b432500dde0a03c6c"
server_key_iv: "c1f23cc23865d80d9f8cc9a0f9dff8d8"
server_key_version: "encryptionkey@0"
I20260620 14:55:36.648273 20666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:55:36.650770 20681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:36.651997 20666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:55:36.652153 20666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/wal
uuid: "056836de87d04c299cb9501c6ce64954"
format_stamp: "Formatted at 2026-06-20 14:55:36 on dist-test-slave-sms9"
server_key: "4edcc61e67962c9b432500dde0a03c6c"
server_key_iv: "c1f23cc23865d80d9f8cc9a0f9dff8d8"
server_key_version: "encryptionkey@0"
I20260620 14:55:36.652272 20666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:36.666056 20666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:36.669154 20666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:36.669364 20666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:36.676637 20666 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.253:35725
I20260620 14:55:36.676662 20733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.253:35725 every 8 connection(s)
I20260620 14:55:36.677606 20666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
I20260620 14:55:36.678179 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 20666
I20260620 14:55:36.678318 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-1/wal/instance
I20260620 14:55:36.678645 7267 external_mini_cluster.cc:1428] Setting key 64f6ec344dbc06b1690f2af7ca8a1646
I20260620 14:55:36.680591 20734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:36.684496 20734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.24.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.252@KRBTEST.COM" created.
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:36.698231 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.686496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57026 (local address 127.7.24.254:42677)
0620 14:55:36.686628 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.686632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.687633 (+ 1001us) server_negotiation.cc:407] Connection header received
0620 14:55:36.688563 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.688567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.688627 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.688743 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.689880 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.690416 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.691587 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.691922 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.692056 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.694450 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.694454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.694499 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.694531 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.696080 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.696618 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.696622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.696623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.696699 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.697053 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.697056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.697058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.697242 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.697352 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.697888 (+ 536us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.698033 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 14:55:36.699123 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.686890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42677 (local address 127.0.0.1:57026)
0620 14:55:36.687440 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.687482 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:36.688347 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:36.688912 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:36.688919 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:36.689241 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:36.689731 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.689744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.690597 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.690601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.691440 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.691449 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.692095 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.692100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.692270 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.692888 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:36.692905 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:36.694282 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:36.696206 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.696213 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.696228 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.696495 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.696799 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.696802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.696803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.696945 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.697352 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:36.697360 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:36.697627 (+ 267us) client_negotiation.cc:769] Sending connection context
0620 14:55:36.697832 (+ 205us) client_negotiation.cc:240] Negotiation successful
0620 14:55:36.698164 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":127,"threads_started":1}
I20260620 14:55:36.700616 20734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab.
Entry for principal kudu/127.7.24.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab.
I20260620 14:55:36.705706 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.252:34779
--webserver_interface=127.7.24.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab
--principal=kudu/127.7.24.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:42677,127.7.24.253:35725,127.7.24.252:34779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
I20260620 14:55:36.710166 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.701166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35725 (local address 127.0.0.1:50742)
0620 14:55:36.701307 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.701321 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:36.701399 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:36.701976 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:36.701978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:36.702167 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:36.702258 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.702263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.703428 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.703431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.703969 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.703977 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.704675 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.704679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.704802 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.705356 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:36.705367 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:36.706639 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:36.708581 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.708586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.708588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.708778 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.709081 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.709083 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.709085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.709169 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.709635 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:36.709638 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:36.709723 (+ 85us) client_negotiation.cc:769] Sending connection context
0620 14:55:36.709843 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 14:55:36.709975 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 14:55:36.710211 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.701283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50742 (local address 127.7.24.253:35725)
0620 14:55:36.701633 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.701643 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.701662 (+ 19us) server_negotiation.cc:407] Connection header received
0620 14:55:36.701698 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.701702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.701854 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.701962 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.702358 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.703324 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.704102 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.704550 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.704650 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.706776 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.706785 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.706795 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.706815 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.708447 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.708889 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.708895 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.708897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.708976 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.709259 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.709264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.709265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.709524 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.709665 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.709925 (+ 260us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.710063 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":147,"threads_started":1}
I20260620 14:55:36.712380 20734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:55:36.713146 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.712854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43832)
0620 14:55:36.712984 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.713081 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260620 14:55:36.713357 20684 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
I20260620 14:55:36.713903 20659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } attempt: 3
I20260620 14:55:36.714263 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.714003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43838)
0620 14:55:36.714114 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.714189 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 14:55:36.714664 20734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:34779: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
I20260620 14:55:36.724102 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.714382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35725 (local address 127.0.0.1:50748)
0620 14:55:36.714543 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.714559 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:36.714648 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:36.715044 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:36.715046 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:36.715199 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:36.715706 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.715713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.717485 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.717488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.718031 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.718038 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.718671 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.718674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.718763 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.719326 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:36.719339 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:36.720791 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:36.722679 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.722682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.722684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.722876 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.723154 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.723156 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.723158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.723230 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.723699 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:36.723701 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:36.723784 (+ 83us) client_negotiation.cc:769] Sending connection context
0620 14:55:36.723865 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 14:55:36.723972 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 14:55:36.724215 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.714667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50748 (local address 127.7.24.253:35725)
0620 14:55:36.714796 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.714802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.714821 (+ 19us) server_negotiation.cc:407] Connection header received
0620 14:55:36.714865 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.714871 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.714927 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.715026 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.715833 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.717374 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.718137 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.718551 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.718654 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.720930 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.720935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.720937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.720965 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.722559 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.722966 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.722969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.722971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.723057 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.723326 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.723331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.723335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.723601 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.723705 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.723939 (+ 234us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.724056 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:55:36.724937 20659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:55:36.725627 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.725391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43846)
0620 14:55:36.725507 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.725559 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260620 14:55:36.726295 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.726059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43856)
0620 14:55:36.726195 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.726249 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260620 14:55:36.726418 20659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:34779: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
I20260620 14:55:36.757582 20659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } attempt: 1
I20260620 14:55:36.758582 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.758161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43860)
0620 14:55:36.758342 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.758407 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260620 14:55:36.759305 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.759095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43864)
0620 14:55:36.759204 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.759256 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260620 14:55:36.759428 20659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:34779: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
I20260620 14:55:36.763938 20734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } attempt: 1
I20260620 14:55:36.764678 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.764379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43880)
0620 14:55:36.764533 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.764598 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260620 14:55:36.765384 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.765164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43896)
0620 14:55:36.765275 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.765326 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 14:55:36.765579 20734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:34779: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
W20260620 14:55:36.824916 20740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:36.825193 20740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:36.825237 20740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:36.829391 20740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:55:36.829460 20740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:36.829481 20740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:36.829499 20740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:55:36.829555 20740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:55:36.833781 20734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } attempt: 2
I20260620 14:55:36.834527 20659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } attempt: 2
I20260620 14:55:36.834654 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.834301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43906)
0620 14:55:36.834490 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.834560 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 14:55:36.835296 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.834990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43918)
0620 14:55:36.835150 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.835212 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260620 14:55:36.835700 20740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:42677,127.7.24.253:35725,127.7.24.252:34779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab
--principal=kudu/127.7.24.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.252:34779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.24.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:36.835875 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.835601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43930)
0620 14:55:36.835735 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.835782 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 14:55:36.836083 20734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:34779: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
I20260620 14:55:36.836390 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.836172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43946)
0620 14:55:36.836272 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.836325 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 14:55:36.836525 20659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:34779: Network error: Client connection negotiation failed: client connection to 127.7.24.252:34779: connect: Connection refused (error 111)
I20260620 14:55:36.837026 20740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:36.838086 20740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:36.847209 20746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:36.847376 20740 server_base.cc:1034] running on GCE node
W20260620 14:55:36.847204 20747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:36.847218 20749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:36.848152 20740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:36.849393 20740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:36.850582 20740 hybrid_clock.cc:648] HybridClock initialized: now 1781967336850562 us; error 34 us; skew 500 ppm
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:36.853437 20740 init.cc:377] Logged in from keytab as kudu/127.7.24.252@KRBTEST.COM (short username kudu)
I20260620 14:55:36.854374 20740 webserver.cc:460] Webserver started at http://127.7.24.252:37505/ using document root <none> and password file <none>
I20260620 14:55:36.854974 20740 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:36.855070 20740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:36.855378 20740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:55:36.857623 20740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/data/instance:
uuid: "33b2eb7f4b3c488db99168e2671401ec"
format_stamp: "Formatted at 2026-06-20 14:55:36 on dist-test-slave-sms9"
server_key: "88078294f6f1eb3ea35fd17436e2e70a"
server_key_iv: "8bab32a4959969ad680571b6f09773b4"
server_key_version: "encryptionkey@0"
I20260620 14:55:36.858201 20740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/wal/instance:
uuid: "33b2eb7f4b3c488db99168e2671401ec"
format_stamp: "Formatted at 2026-06-20 14:55:36 on dist-test-slave-sms9"
server_key: "88078294f6f1eb3ea35fd17436e2e70a"
server_key_iv: "8bab32a4959969ad680571b6f09773b4"
server_key_version: "encryptionkey@0"
I20260620 14:55:36.862049 20740 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260620 14:55:36.864656 20756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:36.865970 20740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:55:36.866209 20740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/wal
uuid: "33b2eb7f4b3c488db99168e2671401ec"
format_stamp: "Formatted at 2026-06-20 14:55:36 on dist-test-slave-sms9"
server_key: "88078294f6f1eb3ea35fd17436e2e70a"
server_key_iv: "8bab32a4959969ad680571b6f09773b4"
server_key_version: "encryptionkey@0"
I20260620 14:55:36.866387 20740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:36.900579 20740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:36.904083 20740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:36.904304 20740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:36.911263 20740 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.252:34779
I20260620 14:55:36.911283 20808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.252:34779 every 8 connection(s)
I20260620 14:55:36.912222 20740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
I20260620 14:55:36.912906 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 20740
I20260620 14:55:36.913050 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781967176660325-7267-0/minicluster-data/master-2/wal/instance
I20260620 14:55:36.913323 7267 external_mini_cluster.cc:1428] Setting key a22da8bedcdbc1148975fb5e1cc8cd20
I20260620 14:55:36.915386 20809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:36.918959 20809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:36.927256 20734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } attempt: 3
I20260620 14:55:36.927719 20812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.914735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.7.24.252:34779)
0620 14:55:36.915134 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.915150 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.915185 (+ 35us) server_negotiation.cc:407] Connection header received
0620 14:55:36.915869 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.915886 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.916206 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.916531 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.917309 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.918286 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.919173 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.919698 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.919865 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.922782 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.922787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.922796 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.922821 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.924551 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.925044 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.925049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.925053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.925226 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.925573 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.925577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.925578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.926073 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.926247 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.926605 (+ 358us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.926870 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":135,"threads_started":1}
I20260620 14:55:36.931439 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.920630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57038 (local address 127.7.24.254:42677)
0620 14:55:36.920791 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.920796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.921206 (+ 410us) server_negotiation.cc:407] Connection header received
0620 14:55:36.921347 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.921350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.921396 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.921467 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.922204 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.922917 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.924235 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.924556 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.924687 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.927118 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.927122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.927124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.927148 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.929062 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.929589 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.929594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.929598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.929688 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.930332 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.930337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.930338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.930672 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.930781 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.931149 (+ 368us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.931261 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 14:55:36.931464 20813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.920557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:42677 (local address 127.0.0.1:57038)
0620 14:55:36.921037 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.921104 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:36.921248 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:36.921552 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:36.921560 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:36.921905 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:36.922027 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.922033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.923036 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.923043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.924076 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.924098 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.924693 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.924696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.924793 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.925234 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:36.925248 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:36.926969 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:36.929188 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.929195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.929200 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.929470 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.929798 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.929801 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.929803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.929902 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.930779 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:36.930786 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:36.930914 (+ 128us) client_negotiation.cc:769] Sending connection context
0620 14:55:36.931075 (+ 161us) client_negotiation.cc:240] Negotiation successful
0620 14:55:36.931271 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":162,"threads_started":1}
I20260620 14:55:36.933019 20809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
I20260620 14:55:36.935680 20659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } attempt: 3
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:36.937404 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.927734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43960)
0620 14:55:36.927877 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.927895 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:36.927984 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:36.929160 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:36.929163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:36.929351 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:36.929450 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.929455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.930348 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.930354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.931249 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.931260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.931870 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.931874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.931965 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.932360 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:36.932374 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:36.933739 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:36.935685 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.935688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.935690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.935954 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.936261 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.936264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.936266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.936367 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.936846 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:36.936849 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:36.936963 (+ 114us) client_negotiation.cc:769] Sending connection context
0620 14:55:36.937065 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 14:55:36.937203 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260620 14:55:36.937920 20812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.927783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.7.24.252:34779)
0620 14:55:36.928958 (+ 1175us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.928961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.928972 (+ 11us) server_negotiation.cc:407] Connection header received
0620 14:55:36.928998 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.929000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.929030 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.929139 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.929565 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.930216 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.931380 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.931743 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.931841 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.933883 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.933889 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.933893 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.933920 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.935558 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.936068 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.936071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.936073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.936155 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.936478 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.936483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.936486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.936706 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.936800 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.937630 (+ 830us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.937735 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1127,"thread_start_us":96,"threads_started":1}
I20260620 14:55:36.940636 20815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.930267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.7.24.254:42677)
0620 14:55:36.930665 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.930675 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.930689 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:55:36.930727 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.930730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.930772 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.930862 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.931604 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.932403 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.933642 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.934011 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.934144 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.937006 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.937010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.937013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.937040 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.939013 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.939432 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.939437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.939439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.939521 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.939800 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.939803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.939805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.939978 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.940187 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.940323 (+ 136us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.940439 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":101,"threads_started":1}
I20260620 14:55:36.941475 20734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954: Bootstrap starting.
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
I20260620 14:55:36.945088 20734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:36.946079 20734 log.cc:824] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:36.947050 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.936031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43968)
0620 14:55:36.936305 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.936322 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:36.936399 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:36.936671 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:36.936673 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:36.936850 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:36.936936 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.936942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.939041 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.939044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.939769 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.939781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.940432 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.940436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.940531 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.941116 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:36.941130 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:36.942404 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:36.944972 (+ 2568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.944976 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.944978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.945231 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.945531 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.945535 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.945537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.945633 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.946486 (+ 853us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:36.946490 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:36.946604 (+ 114us) client_negotiation.cc:769] Sending connection context
0620 14:55:36.946712 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 14:55:36.946853 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260620 14:55:36.947593 20814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.936268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43968 (local address 127.7.24.252:34779)
0620 14:55:36.936375 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.936386 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.936398 (+ 12us) server_negotiation.cc:407] Connection header received
0620 14:55:36.936506 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.936509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.936554 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.936654 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.937078 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.938911 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.939928 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.940286 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.941664 (+ 1378us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.942598 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.942602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.942605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.942629 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.944843 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.945323 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.945329 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.945332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.945423 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.945755 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.945760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.945763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.946014 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.946420 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.947261 (+ 841us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.947377 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":39}
I20260620 14:55:36.948323 20734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954: No bootstrap required, opened a new log
I20260620 14:55:36.949239 20813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.933456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:35725 (local address 127.0.0.1:50762)
0620 14:55:36.933841 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.933865 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:36.933930 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:36.934543 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:36.934548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:36.934743 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:36.935207 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.935215 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.936756 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.936761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.937829 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.937840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.938611 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.938614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.942833 (+ 4219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.943237 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:36.943257 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:36.944969 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:36.947360 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.947376 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.947378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.947607 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.948156 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.948160 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.948162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.948247 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.948670 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:36.948674 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:36.948770 (+ 96us) client_negotiation.cc:769] Sending connection context
0620 14:55:36.948877 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 14:55:36.949035 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":238}
I20260620 14:55:36.949712 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.933697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.7.24.253:35725)
0620 14:55:36.933830 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.933837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.933902 (+ 65us) server_negotiation.cc:407] Connection header received
0620 14:55:36.934003 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.934006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.934056 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.934367 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.935343 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.935973 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.937963 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.938334 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.938408 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.945127 (+ 6719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.945132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.945135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.945163 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.946997 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.947969 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.947973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.947975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.948061 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.948344 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.948348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.948350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.948565 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.948670 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.949330 (+ 660us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.949454 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 14:55:36.950302 20809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
I20260620 14:55:36.952592 20734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } }
I20260620 14:55:36.952808 20734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:36.952860 20734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 056836de87d04c299cb9501c6ce64954, State: Initialized, Role: FOLLOWER
I20260620 14:55:36.953383 20734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } }
I20260620 14:55:36.953928 20818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } } }
I20260620 14:55:36.954090 20818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:55:36.955493 20816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.943629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.7.24.253:35725)
0620 14:55:36.944124 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.944136 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.944151 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:55:36.944966 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.944972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.945034 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.945135 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.946229 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.947014 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.948143 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.948473 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.948566 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.951490 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.951493 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.951495 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.951514 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.953288 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.953695 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.953700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.953702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.953790 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.954123 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.954126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.954129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.954345 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.954567 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.955154 (+ 587us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.955298 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":94,"threads_started":1}
Jun 20 14:55:36 dist-test-slave-sms9 krb5kdc[20578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
I20260620 14:55:36.957779 20659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd: Bootstrap starting.
I20260620 14:55:36.958590 20734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:55:36.959263 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:55:36.960667 20659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:36.961361 20814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.951030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43970 (local address 127.7.24.252:34779)
0620 14:55:36.951137 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.951143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.951159 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:55:36.951192 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.951195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.951239 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.951324 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.951736 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.953149 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.954372 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.954715 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.954810 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.957016 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.957020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.957022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.957042 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.959182 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.959759 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.959762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.959764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.959845 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.960137 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.960140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.960142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.960359 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.960440 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.961153 (+ 713us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.961253 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 14:55:36.961582 20659 log.cc:824] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:36.961977 20813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.950821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:34779 (local address 127.0.0.1:43970)
0620 14:55:36.951013 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:36.951029 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:36.951116 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:36.951352 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:36.951355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:36.951518 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:36.951607 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.951614 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.953263 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.953267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.954248 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:36.954259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.954917 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.954921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:36.955004 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.955458 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:36.955471 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:36.956916 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:36.959375 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.959378 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.959380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.959645 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.959954 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:36.959959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:36.959971 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:36.960050 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:36.960508 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:36.960513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:36.960612 (+ 99us) client_negotiation.cc:769] Sending connection context
0620 14:55:36.961678 (+ 1066us) client_negotiation.cc:240] Negotiation successful
0620 14:55:36.961810 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260620 14:55:36.964365 20659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd: No bootstrap required, opened a new log
I20260620 14:55:36.964859 20809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec: Bootstrap starting.
W20260620 14:55:36.965401 20833 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:55:36.965485 20833 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:55:36.968466 20659 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } }
I20260620 14:55:36.968650 20659 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:36.968739 20659 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a49fe17cec4c5c8f24000b1939befd, State: Initialized, Role: FOLLOWER
I20260620 14:55:36.969192 20659 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } }
I20260620 14:55:36.969687 20839 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } } }
I20260620 14:55:36.969815 20839 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:55:36.970742 20659 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:55:36.967455 20666 master_runner.cc:186] Error getting master registration for 127.7.24.254:42677: OK, instance_id { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" instance_seqno: 1781967336467688 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 14:55:36.970953 20666 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 14:55:36.971719 20809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:36.972628 20809 log.cc:824] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec: Log is configured to *not* fsync() on all Append() calls
W20260620 14:55:36.976207 20850 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:55:36.976290 20850 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:55:36.976781 20818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:55:36.976943 20818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } }
I20260620 14:55:36.978336 20633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "056836de87d04c299cb9501c6ce64954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a49fe17cec4c5c8f24000b1939befd" is_pre_election: true
I20260620 14:55:36.978569 20818 leader_election.cc:288] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7a49fe17cec4c5c8f24000b1939befd (127.7.24.254:42677), 33b2eb7f4b3c488db99168e2671401ec (127.7.24.252:34779)
I20260620 14:55:36.978658 20633 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 056836de87d04c299cb9501c6ce64954 in term 0.
I20260620 14:55:36.980892 20809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec: No bootstrap required, opened a new log
I20260620 14:55:36.982333 20685 leader_election.cc:302] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 056836de87d04c299cb9501c6ce64954, d7a49fe17cec4c5c8f24000b1939befd; no voters:
I20260620 14:55:36.980043 20591 master_runner.cc:186] Error getting master registration for 127.7.24.252:34779: OK, instance_id { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" instance_seqno: 1781967336909272 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 14:55:36.984472 20591 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 14:55:36.985019 20809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } }
I20260620 14:55:36.985294 20809 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:36.985349 20809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b2eb7f4b3c488db99168e2671401ec, State: Initialized, Role: FOLLOWER
I20260620 14:55:36.985708 20818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:55:36.985893 20818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:55:36.986023 20818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:36.988113 20818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } }
I20260620 14:55:36.988827 20633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "056836de87d04c299cb9501c6ce64954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a49fe17cec4c5c8f24000b1939befd"
I20260620 14:55:36.988974 20633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:36.989952 20815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.966402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57052 (local address 127.7.24.254:42677)
0620 14:55:36.966533 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.966537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.973141 (+ 6604us) server_negotiation.cc:407] Connection header received
0620 14:55:36.974779 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.974782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.974827 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.974877 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.976758 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.977464 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.979621 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.979969 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.980045 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.984933 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.984937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.984940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.984967 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.986959 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.988393 (+ 1434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.988399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.988403 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.988498 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.988828 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.988832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.988833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.989110 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.989248 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.989621 (+ 373us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.989778 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 14:55:36.990233 20633 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 056836de87d04c299cb9501c6ce64954 in term 1.
I20260620 14:55:36.990569 20685 leader_election.cc:302] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 056836de87d04c299cb9501c6ce64954, d7a49fe17cec4c5c8f24000b1939befd; no voters:
I20260620 14:55:36.991233 20818 leader_election.cc:288] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [CANDIDATE]: Term 1 election: Requested vote from peers d7a49fe17cec4c5c8f24000b1939befd (127.7.24.254:42677), 33b2eb7f4b3c488db99168e2671401ec (127.7.24.252:34779)
I20260620 14:55:36.991348 20818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:55:36.991428 20818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [term 1 LEADER]: Becoming Leader. State: Replica: 056836de87d04c299cb9501c6ce64954, State: Running, Role: LEADER
I20260620 14:55:36.991554 20814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.965995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43978 (local address 127.7.24.252:34779)
0620 14:55:36.966108 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.966113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.973346 (+ 7233us) server_negotiation.cc:407] Connection header received
0620 14:55:36.973642 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.973645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.973696 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.973749 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.975873 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.976618 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.980598 (+ 3980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.980915 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.980994 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.985344 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.985347 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.985350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.985377 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.987769 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.989030 (+ 1261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.989034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.989035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.989120 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.989871 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.989875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.989877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.990106 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.990258 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.991263 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.991394 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 14:55:36.991791 20818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } }
I20260620 14:55:36.993741 20818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 056836de87d04c299cb9501c6ce64954. Latest consensus state: current_term: 1 leader_uuid: "056836de87d04c299cb9501c6ce64954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } } }
I20260620 14:55:36.993848 20818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:36.994311 20809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } }
I20260620 14:55:36.995751 20809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:55:36.996284 20783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "056836de87d04c299cb9501c6ce64954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b2eb7f4b3c488db99168e2671401ec" is_pre_election: true
I20260620 14:55:36.996538 20816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:36.967101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50784 (local address 127.7.24.253:35725)
0620 14:55:36.970615 (+ 3514us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:36.970618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:36.983252 (+ 12634us) server_negotiation.cc:407] Connection header received
0620 14:55:36.983951 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:36.983954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:36.984006 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:36.984055 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:36.984872 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.985640 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.986911 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:36.987305 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:36.987387 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:36.992197 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:36.992200 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:36.992203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:36.992231 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:36.994017 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.995072 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.995075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.995077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.995169 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:36.995583 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:36.995586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:36.995588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:36.995774 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:36.995931 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:36.996258 (+ 327us) server_negotiation.cc:299] Negotiation successful
0620 14:55:36.996395 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3420}
I20260620 14:55:36.996747 20783 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 056836de87d04c299cb9501c6ce64954 in term 0.
I20260620 14:55:36.997133 20782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "056836de87d04c299cb9501c6ce64954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33b2eb7f4b3c488db99168e2671401ec"
I20260620 14:55:36.997268 20782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:36.998521 20782 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 056836de87d04c299cb9501c6ce64954 in term 1.
I20260620 14:55:37.000000 20856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } } }
I20260620 14:55:37.000110 20856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:55:37.000170 20855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:55:37.001046 20855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260620 14:55:37.002161 20867 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:55:37.002229 20867 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:55:37.013021 20782 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [term 1 FOLLOWER]: Refusing update from remote peer 056836de87d04c299cb9501c6ce64954: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:55:37.013101 20633 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [term 1 FOLLOWER]: Refusing update from remote peer 056836de87d04c299cb9501c6ce64954: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:55:37.013701 20853 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:55:37.014137 20853 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:55:37.018015 20839 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 056836de87d04c299cb9501c6ce64954. Latest consensus state: current_term: 1 leader_uuid: "056836de87d04c299cb9501c6ce64954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } } }
I20260620 14:55:37.018131 20839 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:55:37.018263 20856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 056836de87d04c299cb9501c6ce64954. Latest consensus state: current_term: 1 leader_uuid: "056836de87d04c299cb9501c6ce64954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } } }
I20260620 14:55:37.018369 20856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:55:37.026031 20818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "056836de87d04c299cb9501c6ce64954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } } }
I20260620 14:55:37.026170 20818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:37.027375 20839 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "056836de87d04c299cb9501c6ce64954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } } }
I20260620 14:55:37.027488 20839 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:55:37.027984 20870 mvcc.cc:205] Tried to move back new op lower bound from 7298938212397043712 to 7298938212321951744. Current Snapshot: MvccSnapshot[applied={T|T < 7298938212397043712}]
I20260620 14:55:37.028137 20869 mvcc.cc:205] Tried to move back new op lower bound from 7298938212397043712 to 7298938212321951744. Current Snapshot: MvccSnapshot[applied={T|T < 7298938212397043712}]
I20260620 14:55:37.030754 20853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "056836de87d04c299cb9501c6ce64954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } } }
I20260620 14:55:37.030866 20853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:37.031894 20856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "056836de87d04c299cb9501c6ce64954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a49fe17cec4c5c8f24000b1939befd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42677 } } peers { permanent_uuid: "056836de87d04c299cb9501c6ce64954" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 35725 } } peers { permanent_uuid: "33b2eb7f4b3c488db99168e2671401ec" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 34779 } } }
I20260620 14:55:37.032016 20856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:55:37.034559 20871 mvcc.cc:205] Tried to move back new op lower bound from 7298938212397043712 to 7298938212321951744. Current Snapshot: MvccSnapshot[applied={T|T < 7298938212397043712}]
I20260620 14:55:37.040052 20855 catalog_manager.cc:1347] Generated new cluster ID: a95eb82b5ca3421e8bbb2a730afcc593
I20260620 14:55:37.040130 20855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:55:37.059116 20855 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:55:37.060624 20855 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:55:37.079741 20855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 056836de87d04c299cb9501c6ce64954: Generated new TSK 0
I20260620 14:55:37.080716 20855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:55:37.977615 20850 catalog_manager.cc:1259] Loaded cluster ID: a95eb82b5ca3421e8bbb2a730afcc593
I20260620 14:55:37.977702 20850 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd: loading cluster ID for follower catalog manager: success
I20260620 14:55:37.979364 20850 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd: acquiring CA information for follower catalog manager: success
I20260620 14:55:37.980473 20850 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d7a49fe17cec4c5c8f24000b1939befd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 14:55:38.003564 20867 catalog_manager.cc:1259] Loaded cluster ID: a95eb82b5ca3421e8bbb2a730afcc593
I20260620 14:55:38.003654 20867 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec: loading cluster ID for follower catalog manager: success
I20260620 14:55:38.005308 20867 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec: acquiring CA information for follower catalog manager: success
I20260620 14:55:38.006531 20867 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33b2eb7f4b3c488db99168e2671401ec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 14:55:38.149279 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 20591
I20260620 14:55:38.155862 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 20666
I20260620 14:55:38.161813 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 20740
2026-06-20T14:55:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3058 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:55:38 dist-test-slave-sms9 krb5kdc[20884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:55:38 dist-test-slave-sms9 krb5kdc[20884](info): set up 2 sockets
Jun 20 14:55:38 dist-test-slave-sms9 krb5kdc[20884](info): commencing operation
krb5kdc: starting...
W20260620 14:55:39.215237 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:55:39 dist-test-slave-sms9 krb5kdc[20884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:55:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:55:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:55:39.342620 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:39173
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:40559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:39173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:39.455080 20897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:39.455367 20897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:39.455443 20897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:39.459499 20897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:55:39.459565 20897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:39.459584 20897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:39.459601 20897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:55:39.459661 20897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:55:39.464972 20897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:40559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:39173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:39173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:39.466120 20897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:39.467196 20897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:39.475816 20903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:39.475821 20902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:39.476027 20905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:39.476063 20897 server_base.cc:1034] running on GCE node
I20260620 14:55:39.476670 20897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:39.477746 20897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:39.478926 20897 hybrid_clock.cc:648] HybridClock initialized: now 1781967339478895 us; error 46 us; skew 500 ppm
Jun 20 14:55:39 dist-test-slave-sms9 krb5kdc[20884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967339, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:39.481611 20897 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:55:39.482584 20897 webserver.cc:460] Webserver started at http://127.7.24.254:33593/ using document root <none> and password file <none>
I20260620 14:55:39.483139 20897 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:39.483227 20897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:39.483455 20897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:55:39.485450 20897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "c23aea20d3cf4b7cbcad37f04e921450"
format_stamp: "Formatted at 2026-06-20 14:55:39 on dist-test-slave-sms9"
server_key: "1a8f0a475d37bdc40aec08131f04b7bf"
server_key_iv: "8a28a2645b3644e83fa8dd6a965752c8"
server_key_version: "encryptionkey@0"
I20260620 14:55:39.486037 20897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "c23aea20d3cf4b7cbcad37f04e921450"
format_stamp: "Formatted at 2026-06-20 14:55:39 on dist-test-slave-sms9"
server_key: "1a8f0a475d37bdc40aec08131f04b7bf"
server_key_iv: "8a28a2645b3644e83fa8dd6a965752c8"
server_key_version: "encryptionkey@0"
I20260620 14:55:39.489806 20897 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260620 14:55:39.492250 20912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:39.493466 20897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260620 14:55:39.493664 20897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "c23aea20d3cf4b7cbcad37f04e921450"
format_stamp: "Formatted at 2026-06-20 14:55:39 on dist-test-slave-sms9"
server_key: "1a8f0a475d37bdc40aec08131f04b7bf"
server_key_iv: "8a28a2645b3644e83fa8dd6a965752c8"
server_key_version: "encryptionkey@0"
I20260620 14:55:39.493788 20897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:39.507189 20897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:39.510102 20897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:39.510311 20897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:39.517457 20897 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:39173
I20260620 14:55:39.517477 20964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:39173 every 8 connection(s)
I20260620 14:55:39.518397 20897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:55:39.519073 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 20897
I20260620 14:55:39.519193 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:55:39.519460 7267 external_mini_cluster.cc:1428] Setting key 30a5206d771d97ee20c62239352e9d95
I20260620 14:55:39.521402 20965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:39.527911 20965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450: Bootstrap starting.
Jun 20 14:55:39 dist-test-slave-sms9 krb5kdc[20884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:39.530660 20965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:39.531571 20965 log.cc:824] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:39.533496 20965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450: No bootstrap required, opened a new log
I20260620 14:55:39.533731 20968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:39.520791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48046 (local address 127.7.24.254:39173)
0620 14:55:39.521182 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:39.521205 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:39.521239 (+ 34us) server_negotiation.cc:407] Connection header received
0620 14:55:39.521903 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:39.521920 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:39.522233 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:39.522566 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:39.523400 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:39.524359 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:39.525464 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:39.525901 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:39.526016 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:39.528806 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:39.528812 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:39.528827 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:39.528857 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:39.530805 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:39.531251 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:39.531257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:39.531262 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:39.531403 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:39.531781 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:39.531789 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:39.531790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:39.532163 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:39.532362 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:39.532662 (+ 300us) server_negotiation.cc:299] Negotiation successful
0620 14:55:39.532940 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":137,"threads_started":1}
I20260620 14:55:39.536695 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:55:39.537612 20965 raft_consensus.cc:348] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23aea20d3cf4b7cbcad37f04e921450" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 39173 } }
I20260620 14:55:39.537901 20965 raft_consensus.cc:374] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:39.537962 20965 raft_consensus.cc:729] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c23aea20d3cf4b7cbcad37f04e921450, State: Initialized, Role: FOLLOWER
I20260620 14:55:39.538419 20965 consensus_queue.cc:260] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23aea20d3cf4b7cbcad37f04e921450" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 39173 } }
I20260620 14:55:39.538610 20965 raft_consensus.cc:388] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:55:39.538661 20965 raft_consensus.cc:482] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:55:39.538733 20965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:39.539587 20965 raft_consensus.cc:504] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23aea20d3cf4b7cbcad37f04e921450" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 39173 } }
I20260620 14:55:39.539870 20965 leader_election.cc:302] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c23aea20d3cf4b7cbcad37f04e921450; no voters:
I20260620 14:55:39.540124 20965 leader_election.cc:288] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:55:39.540262 20970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:55:39.540576 20970 raft_consensus.cc:686] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [term 1 LEADER]: Becoming Leader. State: Replica: c23aea20d3cf4b7cbcad37f04e921450, State: Running, Role: LEADER
I20260620 14:55:39.540874 20970 consensus_queue.cc:237] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23aea20d3cf4b7cbcad37f04e921450" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 39173 } }
I20260620 14:55:39.541359 20965 sys_catalog.cc:564] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:55:39.542539 20972 sys_catalog.cc:455] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c23aea20d3cf4b7cbcad37f04e921450. Latest consensus state: current_term: 1 leader_uuid: "c23aea20d3cf4b7cbcad37f04e921450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23aea20d3cf4b7cbcad37f04e921450" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 39173 } } }
I20260620 14:55:39.542685 20972 sys_catalog.cc:458] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:39.542965 20971 sys_catalog.cc:455] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c23aea20d3cf4b7cbcad37f04e921450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c23aea20d3cf4b7cbcad37f04e921450" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 39173 } } }
I20260620 14:55:39.543074 20971 sys_catalog.cc:458] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:39.543167 20979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:55:39.545712 20979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:55:39.551381 20979 catalog_manager.cc:1347] Generated new cluster ID: 5551abc7b87d4bae86f13b849c0dbfec
I20260620 14:55:39.551452 20979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:55:39.566041 20979 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:55:39.567071 20979 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:55:39.576669 20979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c23aea20d3cf4b7cbcad37f04e921450: Generated new TSK 0
I20260620 14:55:39.577359 20979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:55:39.592613 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 20897
2026-06-20T14:55:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1428 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:55:39 dist-test-slave-sms9 krb5kdc[20992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:55:39 dist-test-slave-sms9 krb5kdc[20992](info): set up 2 sockets
Jun 20 14:55:39 dist-test-slave-sms9 krb5kdc[20992](info): commencing operation
krb5kdc: starting...
W20260620 14:55:40.638682 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:55:40 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:55:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:55:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:55:40.763339 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:32837
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:32837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:40.875552 21005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:40.875886 21005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:40.875972 21005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:40.880059 21005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:55:40.880155 21005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:40.880196 21005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:40.880254 21005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:55:40.880292 21005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:55:40.885777 21005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:32837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:32837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:40.886981 21005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:40.888070 21005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:40.896915 21013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:40.896957 21011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:40.897089 21005 server_base.cc:1034] running on GCE node
W20260620 14:55:40.896934 21010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:40.897957 21005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:40.899266 21005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:40.900484 21005 hybrid_clock.cc:648] HybridClock initialized: now 1781967340900464 us; error 33 us; skew 500 ppm
Jun 20 14:55:40 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967340, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:40.908185 21005 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:55:40.909289 21005 webserver.cc:460] Webserver started at http://127.7.24.254:36655/ using document root <none> and password file <none>
I20260620 14:55:40.909896 21005 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:40.909991 21005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:40.910235 21005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:55:40.912498 21005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "7a44cc72cc234d6a8d4899044246b272"
format_stamp: "Formatted at 2026-06-20 14:55:40 on dist-test-slave-sms9"
server_key: "fe713ae040ce30f99d61a659f88c7016"
server_key_iv: "1b04095d3d9f0cb2c7e3bb4530069c0f"
server_key_version: "encryptionkey@0"
I20260620 14:55:40.913089 21005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "7a44cc72cc234d6a8d4899044246b272"
format_stamp: "Formatted at 2026-06-20 14:55:40 on dist-test-slave-sms9"
server_key: "fe713ae040ce30f99d61a659f88c7016"
server_key_iv: "1b04095d3d9f0cb2c7e3bb4530069c0f"
server_key_version: "encryptionkey@0"
I20260620 14:55:40.917042 21005 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260620 14:55:40.919857 21020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:40.921177 21005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 14:55:40.921334 21005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "7a44cc72cc234d6a8d4899044246b272"
format_stamp: "Formatted at 2026-06-20 14:55:40 on dist-test-slave-sms9"
server_key: "fe713ae040ce30f99d61a659f88c7016"
server_key_iv: "1b04095d3d9f0cb2c7e3bb4530069c0f"
server_key_version: "encryptionkey@0"
I20260620 14:55:40.921459 21005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:40.946830 21005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:40.949890 21005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:40.950106 21005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:40.957376 21005 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:32837
I20260620 14:55:40.957726 21072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:32837 every 8 connection(s)
I20260620 14:55:40.958400 21005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:55:40.959582 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 21005
I20260620 14:55:40.959688 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:55:40.959992 7267 external_mini_cluster.cc:1428] Setting key d45b10ca6ae41ad3b74b8c73d2a65a3c
I20260620 14:55:40.961616 21073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:40.968480 21073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Bootstrap starting.
Jun 20 14:55:40 dist-test-slave-sms9 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:40.971063 21073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:40.971961 21073 log.cc:824] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:40.974237 21073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: No bootstrap required, opened a new log
I20260620 14:55:40.974989 21076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:40.961434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58608 (local address 127.7.24.254:32837)
0620 14:55:40.961844 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:40.961862 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:40.961904 (+ 42us) server_negotiation.cc:407] Connection header received
0620 14:55:40.962772 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:40.962794 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:40.963192 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:40.963545 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:40.964206 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:40.965730 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:40.966601 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:40.967305 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:40.967479 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:40.969946 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:40.969953 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:40.969967 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:40.969999 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:40.972032 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:40.972504 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:40.972509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:40.972513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:40.972628 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:40.972913 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:40.972916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:40.972918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:40.973380 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:40.973615 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:40.973765 (+ 150us) server_negotiation.cc:299] Negotiation successful
0620 14:55:40.974056 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
I20260620 14:55:40.978308 21073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:40.978598 21073 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:40.978681 21073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a44cc72cc234d6a8d4899044246b272, State: Initialized, Role: FOLLOWER
I20260620 14:55:40.979131 21073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:40.979312 21073 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:55:40.979377 21073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:55:40.979497 21073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:40.980392 21073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:40.980728 21073 leader_election.cc:302] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a44cc72cc234d6a8d4899044246b272; no voters:
I20260620 14:55:40.981137 21073 leader_election.cc:288] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:55:40.981273 21078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:55:40.981525 21078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 1 LEADER]: Becoming Leader. State: Replica: 7a44cc72cc234d6a8d4899044246b272, State: Running, Role: LEADER
I20260620 14:55:40.981870 21078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:40.982555 21073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:55:40.983639 21079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a44cc72cc234d6a8d4899044246b272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } } }
I20260620 14:55:40.983683 21080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a44cc72cc234d6a8d4899044246b272. Latest consensus state: current_term: 1 leader_uuid: "7a44cc72cc234d6a8d4899044246b272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } } }
I20260620 14:55:40.983865 21079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:40.983867 21080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:40.984308 21087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:55:40.987231 21087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:55:40.992691 21087 catalog_manager.cc:1347] Generated new cluster ID: f40cd1436dc849c3842dc7d456cd6213
I20260620 14:55:40.992766 21087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:55:40.998059 21087 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:55:40.999209 21087 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:55:41.005887 21087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Generated new TSK 0
I20260620 14:55:41.006639 21087 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:55:41.041888 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:32837
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:41.155112 21099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:41.155417 21099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:41.155467 21099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:41.159885 21099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:41.159955 21099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:41.160017 21099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:55:41.165445 21099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:41.166605 21099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:41.167732 21099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:41.177861 21104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:41.177861 21105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:41.178172 21099 server_base.cc:1034] running on GCE node
W20260620 14:55:41.177866 21107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:41.178822 21099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:41.179607 21099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:41.180779 21099 hybrid_clock.cc:648] HybridClock initialized: now 1781967341180747 us; error 47 us; skew 500 ppm
Jun 20 14:55:41 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967341, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:41.184131 21099 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:55:41.185133 21099 webserver.cc:460] Webserver started at http://127.7.24.193:37875/ using document root <none> and password file <none>
I20260620 14:55:41.185714 21099 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:41.185774 21099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:41.185995 21099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:55:41.188148 21099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "d281af84fa3948838fb0c11ae1729788"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "7e888961baded3022b4df79240d27d47"
server_key_iv: "160c1162677a846736c7cf0fd9ee798c"
server_key_version: "encryptionkey@0"
I20260620 14:55:41.188722 21099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "d281af84fa3948838fb0c11ae1729788"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "7e888961baded3022b4df79240d27d47"
server_key_iv: "160c1162677a846736c7cf0fd9ee798c"
server_key_version: "encryptionkey@0"
I20260620 14:55:41.192540 21099 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:55:41.195019 21114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:41.196440 21099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:55:41.196594 21099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "d281af84fa3948838fb0c11ae1729788"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "7e888961baded3022b4df79240d27d47"
server_key_iv: "160c1162677a846736c7cf0fd9ee798c"
server_key_version: "encryptionkey@0"
I20260620 14:55:41.196720 21099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:41.217000 21099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:41.220662 21099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:41.220901 21099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:41.221554 21099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:41.222435 21099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:41.222576 21099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:41.222653 21099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:41.222694 21099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:41.232599 21099 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:33307
I20260620 14:55:41.232621 21227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:33307 every 8 connection(s)
I20260620 14:55:41.233763 21099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:55:41.238850 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 21099
I20260620 14:55:41.238976 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:55:41.239274 7267 external_mini_cluster.cc:1428] Setting key 54a2a34b90f4f9280167ddb86af8576d
Jun 20 14:55:41 dist-test-slave-sms9 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967341, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:41.248330 21076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.235721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:56207 (local address 127.7.24.254:32837)
0620 14:55:41.235869 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:41.235873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:41.236813 (+ 940us) server_negotiation.cc:407] Connection header received
0620 14:55:41.237730 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:41.237734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:41.237784 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:41.237863 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:41.239140 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.239807 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.241122 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.241364 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.241487 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.244427 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:41.244431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:41.244434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:41.244463 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:41.246128 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:41.246739 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:41.246744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:41.246745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:41.246829 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:41.247205 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:41.247208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:41.247209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:41.247405 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:41.247513 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:41.248024 (+ 511us) server_negotiation.cc:299] Negotiation successful
0620 14:55:41.248163 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:55:41.249166 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.236014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:56207)
0620 14:55:41.236683 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:41.236726 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:41.237517 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:41.238055 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:41.238062 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:41.238413 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:41.238968 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.238987 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.239957 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.239960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.240990 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.241000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.241528 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.241532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.241704 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.242398 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:41.242423 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:41.244238 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:41.246261 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:41.246268 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:41.246285 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:41.246634 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:41.246930 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:41.246932 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:41.246934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:41.247100 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:41.247513 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:41.247521 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:41.247782 (+ 261us) client_negotiation.cc:769] Sending connection context
0620 14:55:41.247969 (+ 187us) client_negotiation.cc:240] Negotiation successful
0620 14:55:41.248322 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":129,"threads_started":1}
I20260620 14:55:41.250802 21228 heartbeater.cc:347] Connected to a master server at 127.7.24.254:32837
I20260620 14:55:41.251096 21228 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:41.251658 21228 heartbeater.cc:511] Master 127.7.24.254:32837 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
I20260620 14:55:41.253445 21037 ts_manager.cc:194] Registered new tserver with Master: d281af84fa3948838fb0c11ae1729788 (127.7.24.193:33307)
I20260620 14:55:41.254844 21037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:56207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:55:41.269493 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:32837
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:41.396202 21233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:41.396596 21233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:41.396698 21233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:41.404887 21233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:41.404954 21233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:41.405015 21233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:55:41.410449 21233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:41.411535 21233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:41.412554 21233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:41.422309 21238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:41.422592 21233 server_base.cc:1034] running on GCE node
W20260620 14:55:41.422432 21241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:41.422309 21239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:41.423158 21233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:41.423810 21233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:41.424990 21233 hybrid_clock.cc:648] HybridClock initialized: now 1781967341424954 us; error 51 us; skew 500 ppm
Jun 20 14:55:41 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967341, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:41.427987 21233 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:55:41.428946 21233 webserver.cc:460] Webserver started at http://127.7.24.194:38583/ using document root <none> and password file <none>
I20260620 14:55:41.429562 21233 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:41.429630 21233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:41.429858 21233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:55:41.432118 21233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "0dbb901be3cf4960be2cdaafad6262e5"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "e7272bcec462c852b1085c95274ecc19"
server_key_iv: "76a2a6e3a40bed961ec47ce383cdcd2b"
server_key_version: "encryptionkey@0"
I20260620 14:55:41.432723 21233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "0dbb901be3cf4960be2cdaafad6262e5"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "e7272bcec462c852b1085c95274ecc19"
server_key_iv: "76a2a6e3a40bed961ec47ce383cdcd2b"
server_key_version: "encryptionkey@0"
I20260620 14:55:41.436745 21233 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:55:41.439284 21248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:41.440615 21233 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:55:41.440770 21233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "0dbb901be3cf4960be2cdaafad6262e5"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "e7272bcec462c852b1085c95274ecc19"
server_key_iv: "76a2a6e3a40bed961ec47ce383cdcd2b"
server_key_version: "encryptionkey@0"
I20260620 14:55:41.440886 21233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:41.467437 21233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:41.470542 21233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:41.470777 21233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:41.471370 21233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:41.472214 21233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:41.472287 21233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:41.472359 21233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:41.472404 21233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:41.481405 21233 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:33419
I20260620 14:55:41.481431 21361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:33419 every 8 connection(s)
I20260620 14:55:41.482308 21233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:55:41.486429 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 21233
I20260620 14:55:41.486568 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:55:41.486878 7267 external_mini_cluster.cc:1428] Setting key cd0d01e4ee48e2789b2276bf0d64e633
Jun 20 14:55:41 dist-test-slave-sms9 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967341, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:55:41.497491 21076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.484296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:50611 (local address 127.7.24.254:32837)
0620 14:55:41.484479 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:41.484483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:41.485541 (+ 1058us) server_negotiation.cc:407] Connection header received
0620 14:55:41.486424 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:41.486428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:41.486531 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:41.486626 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:41.487985 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.488730 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.490032 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.490255 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.490346 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.493230 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:41.493234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:41.493236 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:41.493259 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:41.494812 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:41.495341 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:41.495346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:41.495348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:41.495430 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:41.495936 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:41.495941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:41.495946 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:41.496209 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:41.496312 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:41.497036 (+ 724us) server_negotiation.cc:299] Negotiation successful
0620 14:55:41.497210 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:55:41.498615 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.484753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:50611)
0620 14:55:41.485376 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:41.485456 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:41.486194 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:41.486818 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:41.486824 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:41.487219 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:41.487839 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.487859 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.488857 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.488860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.489882 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.489897 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.490419 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.490426 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.490677 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.491291 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:41.491316 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:41.493060 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:41.494928 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:41.494934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:41.494949 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:41.495222 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:41.495609 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:41.495612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:41.495614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:41.495796 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:41.496341 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:41.496347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:41.496705 (+ 358us) client_negotiation.cc:769] Sending connection context
0620 14:55:41.496948 (+ 243us) client_negotiation.cc:240] Negotiation successful
0620 14:55:41.497329 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
I20260620 14:55:41.501480 21362 heartbeater.cc:347] Connected to a master server at 127.7.24.254:32837
I20260620 14:55:41.501868 21362 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:41.502616 21362 heartbeater.cc:511] Master 127.7.24.254:32837 requested a full tablet report, sending...
I20260620 14:55:41.508977 21037 ts_manager.cc:194] Registered new tserver with Master: 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419)
I20260620 14:55:41.509732 21037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:50611
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:55:41.519527 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:32837
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:41.629551 21367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:41.629777 21367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:41.629817 21367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:41.633745 21367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:41.633796 21367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:41.633905 21367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:55:41.639029 21367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:41.639987 21367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:41.640934 21367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:41.650403 21372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:41.650369 21373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:41.650393 21375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:41.650434 21367 server_base.cc:1034] running on GCE node
I20260620 14:55:41.651039 21367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:41.651671 21367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:41.652858 21367 hybrid_clock.cc:648] HybridClock initialized: now 1781967341652836 us; error 33 us; skew 500 ppm
Jun 20 14:55:41 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967341, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:41.655629 21367 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:55:41.656425 21367 webserver.cc:460] Webserver started at http://127.7.24.195:36821/ using document root <none> and password file <none>
I20260620 14:55:41.656962 21367 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:41.657068 21367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:41.657290 21367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:55:41.659296 21367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "230f2d6748bd4a41a2731326c02b0f89"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "46858317258a5ed84fb58688e574dcd9"
server_key_iv: "99daaef6a8a9de50c0e8c2808e28ab37"
server_key_version: "encryptionkey@0"
I20260620 14:55:41.659854 21367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "230f2d6748bd4a41a2731326c02b0f89"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "46858317258a5ed84fb58688e574dcd9"
server_key_iv: "99daaef6a8a9de50c0e8c2808e28ab37"
server_key_version: "encryptionkey@0"
I20260620 14:55:41.663417 21367 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:55:41.665678 21382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:41.667042 21367 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:55:41.667174 21367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "230f2d6748bd4a41a2731326c02b0f89"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "46858317258a5ed84fb58688e574dcd9"
server_key_iv: "99daaef6a8a9de50c0e8c2808e28ab37"
server_key_version: "encryptionkey@0"
I20260620 14:55:41.667272 21367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:41.679606 21367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:41.682875 21367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:41.683068 21367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:41.683707 21367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:41.684490 21367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:41.684535 21367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:41.684612 21367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:41.684644 21367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:41.693042 21367 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:36437
I20260620 14:55:41.693094 21495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:36437 every 8 connection(s)
I20260620 14:55:41.693943 21367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:55:41.695771 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 21367
I20260620 14:55:41.695873 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:55:41.696179 7267 external_mini_cluster.cc:1428] Setting key 6cafa93d0fa074f2659faca2cf5ef6f3
Jun 20 14:55:41 dist-test-slave-sms9 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967341, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:41.707749 21076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.695846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:47657 (local address 127.7.24.254:32837)
0620 14:55:41.695998 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:41.696004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:41.697088 (+ 1084us) server_negotiation.cc:407] Connection header received
0620 14:55:41.697961 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:41.697965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:41.698019 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:41.698119 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:41.699367 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.699947 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.701308 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.701573 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.701658 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.704338 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:41.704341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:41.704344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:41.704368 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:41.705609 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:41.706130 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:41.706135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:41.706142 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:41.706215 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:41.706656 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:41.706660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:41.706662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:41.706842 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:41.706952 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:41.707456 (+ 504us) server_negotiation.cc:299] Negotiation successful
0620 14:55:41.707579 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 14:55:41.708580 21498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.696210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:47657)
0620 14:55:41.696942 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:41.696992 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:41.697777 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:41.698330 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:41.698337 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:41.698765 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:41.699237 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.699250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.700089 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.700092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.701176 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.701191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.701734 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.701737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.701902 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.702672 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:41.702693 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:41.704164 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:41.705735 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:41.705741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:41.705755 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:41.706008 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:41.706309 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:41.706314 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:41.706316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:41.706535 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:41.706962 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:41.706971 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:41.707215 (+ 244us) client_negotiation.cc:769] Sending connection context
0620 14:55:41.707398 (+ 183us) client_negotiation.cc:240] Negotiation successful
0620 14:55:41.707723 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":135,"threads_started":1}
I20260620 14:55:41.709728 21496 heartbeater.cc:347] Connected to a master server at 127.7.24.254:32837
I20260620 14:55:41.710042 21496 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:41.710585 21496 heartbeater.cc:511] Master 127.7.24.254:32837 requested a full tablet report, sending...
I20260620 14:55:41.711630 21037 ts_manager.cc:194] Registered new tserver with Master: 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437)
I20260620 14:55:41.712150 21037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:47657
I20260620 14:55:41.722082 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:55:41.731690 21076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.724051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58622 (local address 127.7.24.254:32837)
0620 14:55:41.724154 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:41.724157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:41.724416 (+ 259us) server_negotiation.cc:407] Connection header received
0620 14:55:41.724568 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:41.724571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:41.724613 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:41.724701 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:41.725516 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.726037 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.727027 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.727247 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.727358 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.728674 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:41.728678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:41.728681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:41.728712 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:41.729902 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:41.730335 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:41.730340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:41.730342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:41.730426 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:41.730772 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:41.730777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:41.730778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:41.731002 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:41.731152 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:41.731399 (+ 247us) server_negotiation.cc:299] Negotiation successful
0620 14:55:41.731516 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 14:55:41.734642 21037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:55:41.737655 21037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:55:41.752251 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.746217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33307 (local address 127.0.0.1:34502)
0620 14:55:41.746599 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:41.746644 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:41.746763 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:41.747696 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:41.747704 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:41.747727 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:41.747887 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.747899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.749138 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.749144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.750375 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.750386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.751700 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.751704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.751815 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.751850 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 14:55:41.751983 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 14:55:41.752119 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":100,"threads_started":1}
I20260620 14:55:41.752264 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.746430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.7.24.193:33307)
0620 14:55:41.747177 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:41.747198 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:41.747224 (+ 26us) server_negotiation.cc:407] Connection header received
0620 14:55:41.747301 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:41.747306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:41.747526 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:41.748049 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:41.748233 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.749009 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.750666 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.751573 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.751698 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.751846 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:41.752013 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 14:55:41.752106 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":95,"threads_started":1}
I20260620 14:55:41.752554 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.746964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:33419 (local address 127.0.0.1:57760)
0620 14:55:41.747181 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:41.747194 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:41.747276 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:41.748063 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:41.748069 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:41.748089 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:41.748257 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.748270 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.749736 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.749743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.751020 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.751029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.752156 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.752160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.752244 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.752269 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 14:55:41.752322 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 14:55:41.752395 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":162,"thread_start_us":96,"threads_started":1}
I20260620 14:55:41.752765 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.746426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:36437 (local address 127.0.0.1:57656)
0620 14:55:41.746830 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:41.746848 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:41.746944 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:41.748088 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:41.748090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:41.748102 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:41.748231 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.748252 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.749750 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.749753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.751126 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.751136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.752316 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.752320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.752410 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.752427 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 14:55:41.752491 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 14:55:41.752542 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":151,"spinlock_wait_cycles":896,"thread_start_us":99,"threads_started":1}
I20260620 14:55:41.752902 21511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.747063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.7.24.195:36437)
0620 14:55:41.747702 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:41.747713 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:41.747730 (+ 17us) server_negotiation.cc:407] Connection header received
0620 14:55:41.747792 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:41.747797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:41.747937 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:41.748405 (+ 468us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:41.748551 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.749624 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.751229 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.752000 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.752126 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.752255 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:41.752631 (+ 376us) server_negotiation.cc:299] Negotiation successful
0620 14:55:41.752740 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":147,"threads_started":1}
I20260620 14:55:41.753968 21512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.747259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.7.24.194:33419)
0620 14:55:41.747618 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:41.747629 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:41.747653 (+ 24us) server_negotiation.cc:407] Connection header received
0620 14:55:41.747730 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:41.747735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:41.747901 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:41.748212 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:41.748384 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.749567 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.751160 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.752000 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.753542 (+ 1542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.753678 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:41.753732 (+ 54us) server_negotiation.cc:299] Negotiation successful
0620 14:55:41.753818 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":123,"threads_started":1}
I20260620 14:55:41.755633 21430 tablet_service.cc:1467] Processing CreateTablet for tablet 7d0f9b69854f4ab1bc607ec1033ae4a8 (DEFAULT_TABLE table=test-table [id=cb848243e93848479a082901108d8dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:55:41.755637 21162 tablet_service.cc:1467] Processing CreateTablet for tablet 7d0f9b69854f4ab1bc607ec1033ae4a8 (DEFAULT_TABLE table=test-table [id=cb848243e93848479a082901108d8dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:55:41.756592 21430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d0f9b69854f4ab1bc607ec1033ae4a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:41.756625 21162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d0f9b69854f4ab1bc607ec1033ae4a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:41.756362 21296 tablet_service.cc:1467] Processing CreateTablet for tablet 7d0f9b69854f4ab1bc607ec1033ae4a8 (DEFAULT_TABLE table=test-table [id=cb848243e93848479a082901108d8dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:55:41.757244 21296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d0f9b69854f4ab1bc607ec1033ae4a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:41.762212 21513 tablet_bootstrap.cc:492] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89: Bootstrap starting.
I20260620 14:55:41.764134 21514 tablet_bootstrap.cc:492] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Bootstrap starting.
I20260620 14:55:41.764838 21513 tablet_bootstrap.cc:654] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:41.765554 21513 log.cc:824] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:41.765792 21514 tablet_bootstrap.cc:654] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:41.766582 21514 log.cc:824] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:41.766918 21515 tablet_bootstrap.cc:492] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5: Bootstrap starting.
I20260620 14:55:41.767597 21513 tablet_bootstrap.cc:492] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89: No bootstrap required, opened a new log
I20260620 14:55:41.767815 21513 ts_tablet_manager.cc:1397] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 14:55:41.768610 21514 tablet_bootstrap.cc:492] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: No bootstrap required, opened a new log
I20260620 14:55:41.768800 21514 ts_tablet_manager.cc:1397] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 14:55:41.769667 21515 tablet_bootstrap.cc:654] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:41.770623 21515 log.cc:824] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:41.770954 21513 raft_consensus.cc:348] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } }
I20260620 14:55:41.771247 21513 raft_consensus.cc:374] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:41.771348 21513 raft_consensus.cc:729] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230f2d6748bd4a41a2731326c02b0f89, State: Initialized, Role: FOLLOWER
I20260620 14:55:41.771839 21513 consensus_queue.cc:260] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } }
I20260620 14:55:41.772747 21513 ts_tablet_manager.cc:1428] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260620 14:55:41.772830 21515 tablet_bootstrap.cc:492] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5: No bootstrap required, opened a new log
I20260620 14:55:41.773034 21515 ts_tablet_manager.cc:1397] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 14:55:41.773128 21496 heartbeater.cc:503] Master 127.7.24.254:32837 was elected leader, sending a full tablet report...
I20260620 14:55:41.773891 21514 raft_consensus.cc:348] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } }
I20260620 14:55:41.774166 21514 raft_consensus.cc:374] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:41.774271 21514 raft_consensus.cc:729] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d281af84fa3948838fb0c11ae1729788, State: Initialized, Role: FOLLOWER
I20260620 14:55:41.774758 21514 consensus_queue.cc:260] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } }
I20260620 14:55:41.775481 21228 heartbeater.cc:503] Master 127.7.24.254:32837 was elected leader, sending a full tablet report...
I20260620 14:55:41.775774 21514 ts_tablet_manager.cc:1428] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260620 14:55:41.777259 21515 raft_consensus.cc:348] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } }
I20260620 14:55:41.777559 21515 raft_consensus.cc:374] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:41.777669 21515 raft_consensus.cc:729] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dbb901be3cf4960be2cdaafad6262e5, State: Initialized, Role: FOLLOWER
I20260620 14:55:41.778111 21515 consensus_queue.cc:260] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } }
I20260620 14:55:41.778752 21362 heartbeater.cc:503] Master 127.7.24.254:32837 was elected leader, sending a full tablet report...
I20260620 14:55:41.779924 21515 ts_tablet_manager.cc:1428] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260620 14:55:41.820371 21519 raft_consensus.cc:482] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:55:41.820597 21519 raft_consensus.cc:504] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } }
I20260620 14:55:41.821802 21519 leader_election.cc:288] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d281af84fa3948838fb0c11ae1729788 (127.7.24.193:33307), 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419)
I20260620 14:55:41.826102 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.822229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:39679 (local address 127.7.24.193:33307)
0620 14:55:41.822364 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:41.822370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:41.822386 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:55:41.822443 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:41.822450 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:41.822556 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:41.822645 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:41.823001 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.823688 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.824988 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.825474 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.825570 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.825611 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:41.825848 (+ 237us) server_negotiation.cc:299] Negotiation successful
0620 14:55:41.825945 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 14:55:41.826094 21498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.822016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33307 (local address 127.7.24.195:39679)
0620 14:55:41.822191 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:41.822210 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:41.822314 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:41.822699 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:41.822703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:41.822731 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:41.822875 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.822883 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.823820 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.823824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.824847 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.824862 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.825593 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.825597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.825705 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.825743 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 14:55:41.825819 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 14:55:41.825894 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":20}
I20260620 14:55:41.826272 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.822319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:33419 (local address 127.7.24.195:59219)
0620 14:55:41.822700 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:41.822713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:41.822806 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:41.823111 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:41.823114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:41.823134 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:41.823240 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.823246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.824100 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.824103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.825081 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:41.825092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.825830 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.825834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:41.825941 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.825964 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 14:55:41.826035 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 14:55:41.826094 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":1,"thread_start_us":150,"threads_started":1}
I20260620 14:55:41.826303 21512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.822441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:59219 (local address 127.7.24.194:33419)
0620 14:55:41.822639 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:41.822644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:41.822825 (+ 181us) server_negotiation.cc:407] Connection header received
0620 14:55:41.822943 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:41.822946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:41.823008 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:41.823083 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:41.823350 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.823988 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.825205 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.825691 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.825819 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.825865 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:41.826081 (+ 216us) server_negotiation.cc:299] Negotiation successful
0620 14:55:41.826176 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260620 14:55:41.826769 21182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0f9b69854f4ab1bc607ec1033ae4a8" candidate_uuid: "230f2d6748bd4a41a2731326c02b0f89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d281af84fa3948838fb0c11ae1729788" is_pre_election: true
I20260620 14:55:41.826965 21316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0f9b69854f4ab1bc607ec1033ae4a8" candidate_uuid: "230f2d6748bd4a41a2731326c02b0f89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dbb901be3cf4960be2cdaafad6262e5" is_pre_election: true
I20260620 14:55:41.827145 21182 raft_consensus.cc:2454] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 230f2d6748bd4a41a2731326c02b0f89 in term 0.
I20260620 14:55:41.827240 21316 raft_consensus.cc:2454] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 230f2d6748bd4a41a2731326c02b0f89 in term 0.
I20260620 14:55:41.827666 21384 leader_election.cc:302] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230f2d6748bd4a41a2731326c02b0f89, d281af84fa3948838fb0c11ae1729788; no voters:
I20260620 14:55:41.827977 21519 raft_consensus.cc:2781] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:55:41.828096 21519 raft_consensus.cc:482] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:55:41.828154 21519 raft_consensus.cc:3037] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:41.829643 21519 raft_consensus.cc:504] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } }
I20260620 14:55:41.830035 21519 leader_election.cc:288] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [CANDIDATE]: Term 1 election: Requested vote from peers d281af84fa3948838fb0c11ae1729788 (127.7.24.193:33307), 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419)
I20260620 14:55:41.830377 21182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0f9b69854f4ab1bc607ec1033ae4a8" candidate_uuid: "230f2d6748bd4a41a2731326c02b0f89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d281af84fa3948838fb0c11ae1729788"
I20260620 14:55:41.830544 21316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0f9b69854f4ab1bc607ec1033ae4a8" candidate_uuid: "230f2d6748bd4a41a2731326c02b0f89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dbb901be3cf4960be2cdaafad6262e5"
I20260620 14:55:41.830554 21182 raft_consensus.cc:3037] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:41.830673 21316 raft_consensus.cc:3037] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:41.831907 21316 raft_consensus.cc:2454] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 230f2d6748bd4a41a2731326c02b0f89 in term 1.
I20260620 14:55:41.831907 21182 raft_consensus.cc:2454] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 230f2d6748bd4a41a2731326c02b0f89 in term 1.
I20260620 14:55:41.832201 21384 leader_election.cc:302] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dbb901be3cf4960be2cdaafad6262e5, 230f2d6748bd4a41a2731326c02b0f89; no voters:
I20260620 14:55:41.832371 21519 raft_consensus.cc:2781] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:55:41.832614 21519 raft_consensus.cc:686] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 1 LEADER]: Becoming Leader. State: Replica: 230f2d6748bd4a41a2731326c02b0f89, State: Running, Role: LEADER
I20260620 14:55:41.832983 21519 consensus_queue.cc:237] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } }
I20260620 14:55:41.836380 21037 catalog_manager.cc:5495] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195). New cstate: current_term: 1 leader_uuid: "230f2d6748bd4a41a2731326c02b0f89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } health_report { overall_health: HEALTHY } } }
I20260620 14:55:41.859632 21511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.856242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57658 (local address 127.7.24.195:36437)
0620 14:55:41.856383 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:41.856387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:41.856416 (+ 29us) server_negotiation.cc:407] Connection header received
0620 14:55:41.856486 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:41.856488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:41.856541 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:41.856630 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:55:41.856874 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.857356 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.858217 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:41.858416 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:41.858572 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:41.858812 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:55:41.859210 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:55:41.859302 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:41.859435 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 14:55:41.859486 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 14:55:41.865299 21182 raft_consensus.cc:1261] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Refusing update from remote peer 230f2d6748bd4a41a2731326c02b0f89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:55:41.865299 21316 raft_consensus.cc:1261] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5 [term 1 FOLLOWER]: Refusing update from remote peer 230f2d6748bd4a41a2731326c02b0f89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:55:41.866058 21519 consensus_queue.cc:1035] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:55:41.866312 21523 consensus_queue.cc:1035] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:55:41.881243 21036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:55:41.881390 21036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:55:41.883867 21036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Sending DeleteTablet for 3 replicas of tablet 7d0f9b69854f4ab1bc607ec1033ae4a8
I20260620 14:55:41.884667 21296 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d0f9b69854f4ab1bc607ec1033ae4a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:55:41 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:57760
I20260620 14:55:41.884676 21430 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d0f9b69854f4ab1bc607ec1033ae4a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:55:41 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:57656
I20260620 14:55:41.884718 21162 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d0f9b69854f4ab1bc607ec1033ae4a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:55:41 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:34502
I20260620 14:55:41.885149 21536 tablet_replica.cc:330] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89: stopping tablet replica
I20260620 14:55:41.885363 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 21099
I20260620 14:55:41.885386 21535 tablet_replica.cc:330] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5: stopping tablet replica
I20260620 14:55:41.886270 21535 raft_consensus.cc:2229] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:55:41.886581 21535 raft_consensus.cc:2258] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:55:41.887781 21536 raft_consensus.cc:2229] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:55:41.888021 21535 ts_tablet_manager.cc:1905] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:55:41.888192 21536 raft_consensus.cc:2258] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:55:41.890314 21535 ts_tablet_manager.cc:1918] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:55:41.890429 21535 log.cc:1196] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/7d0f9b69854f4ab1bc607ec1033ae4a8
I20260620 14:55:41.890775 21535 ts_tablet_manager.cc:1939] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 0dbb901be3cf4960be2cdaafad6262e5: Deleting consensus metadata
I20260620 14:55:41.891855 21022 catalog_manager.cc:4841] TS 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419): tablet 7d0f9b69854f4ab1bc607ec1033ae4a8 (table test-table [id=cb848243e93848479a082901108d8dce]) successfully deleted
W20260620 14:55:41.893992 21022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:33307 (error 108)
I20260620 14:55:41.894835 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 21233
I20260620 14:55:41.895740 21536 ts_tablet_manager.cc:1905] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:55:41.897805 21536 ts_tablet_manager.cc:1918] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:55:41.897915 21536 log.cc:1196] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/7d0f9b69854f4ab1bc607ec1033ae4a8
I20260620 14:55:41.898254 21536 ts_tablet_manager.cc:1939] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P 230f2d6748bd4a41a2731326c02b0f89: Deleting consensus metadata
I20260620 14:55:41.899304 21023 catalog_manager.cc:4841] TS 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437): tablet 7d0f9b69854f4ab1bc607ec1033ae4a8 (table test-table [id=cb848243e93848479a082901108d8dce]) successfully deleted
I20260620 14:55:41.899967 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:41.899628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33307 (local address 127.0.0.1:34508)
0620 14:55:41.899801 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:41.899894 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260620 14:55:41.900131 21022 catalog_manager.cc:4584] TS d281af84fa3948838fb0c11ae1729788 (127.7.24.193:33307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d0f9b69854f4ab1bc607ec1033ae4a8: Network error: Client connection negotiation failed: client connection to 127.7.24.193:33307: connect: Connection refused (error 111)
I20260620 14:55:41.900761 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 21367
I20260620 14:55:41.906325 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 21005
I20260620 14:55:41.912802 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:32837
--webserver_interface=127.7.24.254
--webserver_port=36655
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:32837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:42.025038 21539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:42.025312 21539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:42.025386 21539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:42.029454 21539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:55:42.029512 21539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:42.029531 21539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:42.029578 21539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:55:42.029609 21539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:55:42.034866 21539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:32837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:32837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=36655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:42.035936 21539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:42.036949 21539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:42.049752 21548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:42.049832 21545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:42.049899 21546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:42.050050 21539 server_base.cc:1034] running on GCE node
I20260620 14:55:42.050771 21539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:42.051918 21539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:42.053081 21539 hybrid_clock.cc:648] HybridClock initialized: now 1781967342053065 us; error 28 us; skew 500 ppm
Jun 20 14:55:42 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:42.055991 21539 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:55:42.056897 21539 webserver.cc:460] Webserver started at http://127.7.24.254:36655/ using document root <none> and password file <none>
I20260620 14:55:42.057530 21539 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:42.057641 21539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:42.061370 21539 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:55:42.063478 21555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:42.064729 21539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:55:42.064882 21539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "7a44cc72cc234d6a8d4899044246b272"
format_stamp: "Formatted at 2026-06-20 14:55:40 on dist-test-slave-sms9"
server_key: "fe713ae040ce30f99d61a659f88c7016"
server_key_iv: "1b04095d3d9f0cb2c7e3bb4530069c0f"
server_key_version: "encryptionkey@0"
I20260620 14:55:42.065476 21539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:42.083760 21539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:42.087155 21539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:42.087365 21539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:42.094313 21539 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:32837
I20260620 14:55:42.094691 21607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:32837 every 8 connection(s)
I20260620 14:55:42.095463 21539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:55:42.099325 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 21539
I20260620 14:55:42.099807 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:33307
--local_ip_for_outbound_sockets=127.7.24.193
--tserver_master_addrs=127.7.24.254:32837
--webserver_port=37875
--webserver_interface=127.7.24.193
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:55:42.100055 21608 sys_catalog.cc:263] Verifying existing consensus state
I20260620 14:55:42.102313 21608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Bootstrap starting.
I20260620 14:55:42.119596 21608 log.cc:824] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:42.128679 21608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:55:42.129061 21608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Bootstrap complete.
I20260620 14:55:42.133070 21608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:42.133448 21608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a44cc72cc234d6a8d4899044246b272, State: Initialized, Role: FOLLOWER
I20260620 14:55:42.133920 21608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:42.134084 21608 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:55:42.134176 21608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:55:42.134294 21608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 1 FOLLOWER]: Advancing to term 2
I20260620 14:55:42.136152 21608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:42.136477 21608 leader_election.cc:302] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a44cc72cc234d6a8d4899044246b272; no voters:
I20260620 14:55:42.136770 21608 leader_election.cc:288] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 14:55:42.136899 21612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 14:55:42.137149 21612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 2 LEADER]: Becoming Leader. State: Replica: 7a44cc72cc234d6a8d4899044246b272, State: Running, Role: LEADER
I20260620 14:55:42.137508 21612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:42.137959 21608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:55:42.138988 21614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a44cc72cc234d6a8d4899044246b272. Latest consensus state: current_term: 2 leader_uuid: "7a44cc72cc234d6a8d4899044246b272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } } }
I20260620 14:55:42.139117 21614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:42.139465 21621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:55:42.139468 21613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7a44cc72cc234d6a8d4899044246b272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } } }
I20260620 14:55:42.139619 21613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:42.144533 21621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:55:42.145219 21621 catalog_manager.cc:1259] Loaded cluster ID: f40cd1436dc849c3842dc7d456cd6213
I20260620 14:55:42.145318 21621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:55:42.146693 21621 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260620 14:55:42.151204 21621 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260620 14:55:42.156418 21621 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:55:42.157214 21621 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Loaded TSK: 0
I20260620 14:55:42.158452 21621 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 14:55:42.218137 21610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:42.218451 21610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:42.218549 21610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:42.222738 21610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:42.222826 21610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:42.222913 21610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:55:42.228433 21610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:33307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=37875
--tserver_master_addrs=127.7.24.254:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:42.229480 21610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:42.230533 21610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:42.241362 21635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:42.241466 21636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:42.241839 21610 server_base.cc:1034] running on GCE node
W20260620 14:55:42.241362 21638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:42.242416 21610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:42.243057 21610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:42.244249 21610 hybrid_clock.cc:648] HybridClock initialized: now 1781967342244209 us; error 64 us; skew 500 ppm
Jun 20 14:55:42 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:42.247169 21610 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:55:42.248051 21610 webserver.cc:460] Webserver started at http://127.7.24.193:37875/ using document root <none> and password file <none>
I20260620 14:55:42.248648 21610 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:42.248765 21610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:42.252784 21610 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:55:42.255138 21645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:42.256619 21610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:55:42.256747 21610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "d281af84fa3948838fb0c11ae1729788"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "7e888961baded3022b4df79240d27d47"
server_key_iv: "160c1162677a846736c7cf0fd9ee798c"
server_key_version: "encryptionkey@0"
I20260620 14:55:42.257285 21610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:42.282763 21610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:42.285928 21610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:42.286127 21610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:42.286844 21610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:42.288000 21652 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 14:55:42.292119 21610 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 14:55:42.292196 21610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260620 14:55:42.292295 21610 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 14:55:42.293912 21610 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 14:55:42.293982 21610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260620 14:55:42.294272 21652 tablet_bootstrap.cc:492] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Bootstrap starting.
I20260620 14:55:42.304457 21610 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:33307
I20260620 14:55:42.304548 21759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:33307 every 8 connection(s)
I20260620 14:55:42.305428 21610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:55:42.306861 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 21610
I20260620 14:55:42.307376 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:33419
--local_ip_for_outbound_sockets=127.7.24.194
--tserver_master_addrs=127.7.24.254:32837
--webserver_port=38583
--webserver_interface=127.7.24.194
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:55:42.312517 21652 log.cc:824] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Log is configured to *not* fsync() on all Append() calls
Jun 20 14:55:42 dist-test-slave-sms9 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:42.320295 21652 tablet_bootstrap.cc:492] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:55:42.320742 21652 tablet_bootstrap.cc:492] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Bootstrap complete.
I20260620 14:55:42.322095 21652 ts_tablet_manager.cc:1397] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Time spent bootstrapping tablet: real 0.028s user 0.020s sys 0.004s
I20260620 14:55:42.325119 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.307421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:39055)
0620 14:55:42.308098 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:42.308150 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:42.308991 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:42.310448 (+ 1457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:42.310493 (+ 45us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:42.310967 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:42.311580 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.311599 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.313584 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.313589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.314855 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.314866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.315977 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.315981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.316167 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.316867 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:42.316887 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:42.318942 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:42.321650 (+ 2708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:42.321656 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:42.321669 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:42.322010 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:42.322423 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:42.322426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:42.322428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:42.322614 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:42.323338 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:42.323344 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:42.323611 (+ 267us) client_negotiation.cc:769] Sending connection context
0620 14:55:42.323815 (+ 204us) client_negotiation.cc:240] Negotiation successful
0620 14:55:42.324202 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":123,"threads_started":1}
I20260620 14:55:42.325119 21652 raft_consensus.cc:348] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } }
I20260620 14:55:42.326174 21652 raft_consensus.cc:729] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d281af84fa3948838fb0c11ae1729788, State: Initialized, Role: FOLLOWER
I20260620 14:55:42.326308 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.307398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:39055 (local address 127.7.24.254:32837)
0620 14:55:42.308463 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:42.308481 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:42.308525 (+ 44us) server_negotiation.cc:407] Connection header received
0620 14:55:42.309562 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:42.309577 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:42.309966 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:42.310318 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:42.311781 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.313425 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.314991 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.315815 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.316021 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.319162 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:42.319169 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:42.319182 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:42.319213 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:42.321508 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:42.322151 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:42.322158 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:42.322164 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:42.322316 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:42.322730 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:42.322734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:42.322736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:42.323169 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:42.323366 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:42.325312 (+ 1946us) server_negotiation.cc:299] Negotiation successful
0620 14:55:42.325610 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":919,"thread_start_us":110,"threads_started":1}
I20260620 14:55:42.326797 21652 consensus_queue.cc:260] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } }
I20260620 14:55:42.327824 21652 ts_tablet_manager.cc:1428] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:55:42.328706 21760 heartbeater.cc:347] Connected to a master server at 127.7.24.254:32837
I20260620 14:55:42.328955 21760 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:42.329469 21760 heartbeater.cc:511] Master 127.7.24.254:32837 requested a full tablet report, sending...
I20260620 14:55:42.331857 21572 ts_manager.cc:194] Registered new tserver with Master: d281af84fa3948838fb0c11ae1729788 (127.7.24.193:33307)
I20260620 14:55:42.335206 21572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:39055
I20260620 14:55:42.337872 21760 heartbeater.cc:503] Master 127.7.24.254:32837 was elected leader, sending a full tablet report...
Jun 20 14:55:42 dist-test-slave-sms9 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:55:42.345652 21767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.334620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33307 (local address 127.0.0.1:34514)
0620 14:55:42.335265 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:42.335305 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:42.335450 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:42.336165 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:42.336171 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:42.336502 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:42.336607 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.336614 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.337598 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.337601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.338382 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.338396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.339818 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.339822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.339928 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.340490 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:42.340502 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:42.341963 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:42.343932 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:42.343942 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:42.343947 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:42.344174 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:42.344481 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:42.344485 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:42.344487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:42.344576 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:42.345025 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:42.345030 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:42.345158 (+ 128us) client_negotiation.cc:769] Sending connection context
0620 14:55:42.345295 (+ 137us) client_negotiation.cc:240] Negotiation successful
0620 14:55:42.345477 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":113,"threads_started":1}
I20260620 14:55:42.345700 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.334687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34514 (local address 127.7.24.193:33307)
0620 14:55:42.335027 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:42.335038 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:42.335595 (+ 557us) server_negotiation.cc:407] Connection header received
0620 14:55:42.335663 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:42.335668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:42.335852 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:42.336004 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:42.336729 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.337473 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.339162 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.339703 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.339817 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.342115 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:42.342123 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:42.342137 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:42.342164 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:42.343824 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:42.344305 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:42.344310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:42.344312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:42.344384 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:42.344675 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:42.344678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:42.344680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:42.344908 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:42.345053 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:42.345386 (+ 333us) server_negotiation.cc:299] Negotiation successful
0620 14:55:42.345551 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":260,"thread_start_us":86,"threads_started":1}
I20260620 14:55:42.347050 21694 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d0f9b69854f4ab1bc607ec1033ae4a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:55:41 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:34514
I20260620 14:55:42.348019 21769 tablet_replica.cc:330] stopping tablet replica
I20260620 14:55:42.348302 21769 raft_consensus.cc:2229] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:55:42.348541 21769 raft_consensus.cc:2258] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:55:42.349449 21769 ts_tablet_manager.cc:1905] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:55:42.352808 21769 ts_tablet_manager.cc:1918] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:55:42.352916 21769 log.cc:1196] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal/wals/7d0f9b69854f4ab1bc607ec1033ae4a8
I20260620 14:55:42.353214 21769 ts_tablet_manager.cc:1939] T 7d0f9b69854f4ab1bc607ec1033ae4a8 P d281af84fa3948838fb0c11ae1729788: Deleting consensus metadata
I20260620 14:55:42.354796 21557 catalog_manager.cc:4841] TS d281af84fa3948838fb0c11ae1729788 (127.7.24.193:33307): tablet 7d0f9b69854f4ab1bc607ec1033ae4a8 (table test-table [id=cb848243e93848479a082901108d8dce]) successfully deleted
W20260620 14:55:42.427035 21764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:42.427281 21764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:42.427322 21764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:42.431200 21764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:42.431249 21764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:42.431306 21764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:55:42.436440 21764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:33419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=38583
--tserver_master_addrs=127.7.24.254:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:42.437472 21764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:42.438570 21764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:42.448431 21774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:42.448438 21777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:42.448511 21764 server_base.cc:1034] running on GCE node
W20260620 14:55:42.448495 21775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:42.449066 21764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:42.449679 21764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:42.450842 21764 hybrid_clock.cc:648] HybridClock initialized: now 1781967342450828 us; error 25 us; skew 500 ppm
Jun 20 14:55:42 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:42.453480 21764 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:55:42.454312 21764 webserver.cc:460] Webserver started at http://127.7.24.194:38583/ using document root <none> and password file <none>
I20260620 14:55:42.454914 21764 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:42.455026 21764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:42.458962 21764 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:55:42.461061 21784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:42.462272 21764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:55:42.462406 21764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "0dbb901be3cf4960be2cdaafad6262e5"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "e7272bcec462c852b1085c95274ecc19"
server_key_iv: "76a2a6e3a40bed961ec47ce383cdcd2b"
server_key_version: "encryptionkey@0"
I20260620 14:55:42.463006 21764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:42.477017 21764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:42.479876 21764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:42.480084 21764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:42.480659 21764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:42.481468 21764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:42.481539 21764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:42.481603 21764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:42.481647 21764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:42.490737 21764 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:33419
I20260620 14:55:42.490761 21897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:33419 every 8 connection(s)
I20260620 14:55:42.491704 21764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:55:42.496088 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 21764
I20260620 14:55:42.496503 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:36437
--local_ip_for_outbound_sockets=127.7.24.195
--tserver_master_addrs=127.7.24.254:32837
--webserver_port=36821
--webserver_interface=127.7.24.195
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 14:55:42 dist-test-slave-sms9 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:42.506518 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.493565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:33823 (local address 127.7.24.254:32837)
0620 14:55:42.493719 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:42.493723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:42.494732 (+ 1009us) server_negotiation.cc:407] Connection header received
0620 14:55:42.495639 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:42.495642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:42.495707 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:42.495813 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:42.497093 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.497685 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.498949 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.499398 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.499526 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.502007 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:42.502013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:42.502016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:42.502060 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:42.503986 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:42.504667 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:42.504674 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:42.504677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:42.504785 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:42.505155 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:42.505160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:42.505162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:42.505442 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:42.505564 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:42.506087 (+ 523us) server_negotiation.cc:299] Negotiation successful
0620 14:55:42.506257 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 14:55:42.507280 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.493891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:33823)
0620 14:55:42.494588 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:42.494630 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:42.495428 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:42.495990 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:42.495996 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:42.496402 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:42.496941 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.496955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.497848 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.497851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.498827 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.498838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.499573 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.499576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.499721 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.500352 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:42.500368 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:42.501827 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:42.504140 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:42.504146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:42.504159 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:42.504543 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:42.504912 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:42.504914 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:42.504915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:42.505052 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:42.505579 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:42.505584 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:42.505825 (+ 241us) client_negotiation.cc:769] Sending connection context
0620 14:55:42.505998 (+ 173us) client_negotiation.cc:240] Negotiation successful
0620 14:55:42.506269 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":100,"threads_started":1}
I20260620 14:55:42.508595 21898 heartbeater.cc:347] Connected to a master server at 127.7.24.254:32837
I20260620 14:55:42.508847 21898 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:42.509333 21898 heartbeater.cc:511] Master 127.7.24.254:32837 requested a full tablet report, sending...
I20260620 14:55:42.510386 21572 ts_manager.cc:194] Registered new tserver with Master: 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419)
I20260620 14:55:42.511018 21572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:33823
W20260620 14:55:42.609261 21901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:42.609527 21901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:42.609570 21901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:42.613739 21901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:42.613801 21901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:42.613893 21901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:55:42.619266 21901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:36437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=36821
--tserver_master_addrs=127.7.24.254:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:42.620301 21901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:42.621340 21901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:42.630796 21909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:42.630796 21907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:42.630869 21906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:42.630903 21901 server_base.cc:1034] running on GCE node
I20260620 14:55:42.631459 21901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:42.632056 21901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:42.633221 21901 hybrid_clock.cc:648] HybridClock initialized: now 1781967342633201 us; error 39 us; skew 500 ppm
Jun 20 14:55:42 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:42.636000 21901 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:55:42.637020 21901 webserver.cc:460] Webserver started at http://127.7.24.195:36821/ using document root <none> and password file <none>
I20260620 14:55:42.637610 21901 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:42.637722 21901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:42.641848 21901 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:55:42.643976 21916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:42.645282 21901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:55:42.645447 21901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "230f2d6748bd4a41a2731326c02b0f89"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "46858317258a5ed84fb58688e574dcd9"
server_key_iv: "99daaef6a8a9de50c0e8c2808e28ab37"
server_key_version: "encryptionkey@0"
I20260620 14:55:42.645977 21901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:42.688271 21901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:42.691380 21901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:42.691589 21901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:42.692173 21901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:42.692978 21901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:42.693054 21901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:42.693142 21901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:42.693188 21901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:42.702225 21901 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:36437
I20260620 14:55:42.702252 22029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:36437 every 8 connection(s)
I20260620 14:55:42.703251 21901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:55:42.703634 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 21901
I20260620 14:55:42.713197 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.704427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.7.24.254:32837)
0620 14:55:42.704570 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:42.704574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:42.704768 (+ 194us) server_negotiation.cc:407] Connection header received
0620 14:55:42.704899 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:42.704904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:42.704970 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:42.705053 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:42.705868 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.706650 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.707851 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.708056 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.708131 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.709701 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:42.709705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:42.709708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:42.709732 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:42.711479 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:42.711965 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:42.711967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:42.711969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:42.712054 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:42.712354 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:42.712360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:42.712362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:42.712562 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:42.712705 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:42.712948 (+ 243us) server_negotiation.cc:299] Negotiation successful
0620 14:55:42.713047 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Jun 20 14:55:42 dist-test-slave-sms9 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:55:42.718135 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.705145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:47675 (local address 127.7.24.254:32837)
0620 14:55:42.705380 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:42.705384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:42.706435 (+ 1051us) server_negotiation.cc:407] Connection header received
0620 14:55:42.707567 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:42.707570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:42.707610 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:42.707682 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:42.709253 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.709824 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.711112 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.711489 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.711596 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.714620 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:42.714624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:42.714626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:42.714649 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:42.716153 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:42.716664 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:42.716667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:42.716669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:42.716748 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:42.717094 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:42.717097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:42.717099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:42.717270 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:42.717375 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:42.717857 (+ 482us) server_negotiation.cc:299] Negotiation successful
0620 14:55:42.717971 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260620 14:55:42.718997 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.705630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:47675)
0620 14:55:42.706296 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:42.706335 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:42.707332 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:42.707904 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:42.707914 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:42.708369 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:42.709097 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.709119 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.709977 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.709980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.710983 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.710993 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.711651 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.711684 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.711931 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.712524 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:42.712540 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:42.714415 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:42.716278 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:42.716283 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:42.716298 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:42.716556 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:42.716846 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:42.716849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:42.716851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:42.716986 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:42.717373 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:42.717380 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:42.717622 (+ 242us) client_negotiation.cc:769] Sending connection context
0620 14:55:42.717804 (+ 182us) client_negotiation.cc:240] Negotiation successful
0620 14:55:42.718131 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":153,"threads_started":1}
I20260620 14:55:42.720052 22030 heartbeater.cc:347] Connected to a master server at 127.7.24.254:32837
I20260620 14:55:42.720402 22030 heartbeater.cc:464] Registering TS with master...
I20260620 14:55:42.720894 22030 heartbeater.cc:511] Master 127.7.24.254:32837 requested a full tablet report, sending...
I20260620 14:55:42.721925 21572 ts_manager.cc:194] Registered new tserver with Master: 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437)
I20260620 14:55:42.722576 21572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:47675
I20260620 14:55:42.727185 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:55:42.731705 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.727825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58654 (local address 127.7.24.254:32837)
0620 14:55:42.728009 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:42.728017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:42.728203 (+ 186us) server_negotiation.cc:407] Connection header received
0620 14:55:42.728318 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:42.728321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:42.728427 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:42.728518 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:55:42.728724 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.729283 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.730187 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.730450 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.730625 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.730923 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:55:42.731304 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:55:42.731410 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:42.731475 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 14:55:42.731527 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260620 14:55:42.732350 21572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:55:42.735703 21572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:55:42.744052 21694 tablet_service.cc:1467] Processing CreateTablet for tablet 7d8c70bf5474462b93a4bc52f1267708 (DEFAULT_TABLE table=test-table [id=0ff693def0624345b5f360ed2960e61f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:55:42.744748 21694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d8c70bf5474462b93a4bc52f1267708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:42.747732 21652 tablet_bootstrap.cc:492] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: Bootstrap starting.
I20260620 14:55:42.748066 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.743011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:33419 (local address 127.0.0.1:57762)
0620 14:55:42.743523 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:42.743538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:42.743633 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:42.744159 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:42.744161 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:42.744180 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:42.744269 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.744275 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.745765 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.745767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.746655 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.746664 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.747656 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.747660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.747740 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.747780 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 14:55:42.747855 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 14:55:42.747926 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":115,"threads_started":1}
I20260620 14:55:42.748116 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.743235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57762 (local address 127.7.24.194:33419)
0620 14:55:42.743722 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:42.743733 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:42.743757 (+ 24us) server_negotiation.cc:407] Connection header received
0620 14:55:42.743829 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:42.743833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:42.744010 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:42.744413 (+ 403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:42.744580 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.745618 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.746776 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.747528 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.747655 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.747805 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:42.747885 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 14:55:42.747982 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":108,"threads_started":1}
I20260620 14:55:42.748382 21767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.742723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:36437 (local address 127.0.0.1:57674)
0620 14:55:42.742863 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:42.742883 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:42.743002 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:42.743832 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:42.743835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:42.743852 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:42.743963 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.743969 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.745408 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.745412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.746764 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:42.746776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.747991 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.747995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:42.748096 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.748122 (+ 26us) client_negotiation.cc:769] Sending connection context
0620 14:55:42.748177 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 14:55:42.748256 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260620 14:55:42.748746 21652 tablet_bootstrap.cc:654] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:42.749766 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:42.742919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57674 (local address 127.7.24.195:36437)
0620 14:55:42.743312 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:42.743322 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:42.743341 (+ 19us) server_negotiation.cc:407] Connection header received
0620 14:55:42.743451 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:42.743456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:42.743682 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:42.743857 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:42.744096 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.745249 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.746921 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:42.747838 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:42.748781 (+ 943us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:42.748942 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:42.749008 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 14:55:42.749132 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":170,"threads_started":1}
I20260620 14:55:42.750253 21652 tablet_bootstrap.cc:492] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: No bootstrap required, opened a new log
I20260620 14:55:42.750342 21652 ts_tablet_manager.cc:1397] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 14:55:42.750815 21652 raft_consensus.cc:348] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:42.750947 21652 raft_consensus.cc:374] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:42.750988 21652 raft_consensus.cc:729] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d281af84fa3948838fb0c11ae1729788, State: Initialized, Role: FOLLOWER
I20260620 14:55:42.751180 21652 consensus_queue.cc:260] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:42.751448 21652 ts_tablet_manager.cc:1428] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 14:55:42.751644 21832 tablet_service.cc:1467] Processing CreateTablet for tablet 7d8c70bf5474462b93a4bc52f1267708 (DEFAULT_TABLE table=test-table [id=0ff693def0624345b5f360ed2960e61f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:55:42.752447 21964 tablet_service.cc:1467] Processing CreateTablet for tablet 7d8c70bf5474462b93a4bc52f1267708 (DEFAULT_TABLE table=test-table [id=0ff693def0624345b5f360ed2960e61f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:55:42.752849 21832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d8c70bf5474462b93a4bc52f1267708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:42.753367 21964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d8c70bf5474462b93a4bc52f1267708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:55:42.759737 22043 tablet_bootstrap.cc:492] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: Bootstrap starting.
I20260620 14:55:42.760411 22044 tablet_bootstrap.cc:492] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: Bootstrap starting.
I20260620 14:55:42.762269 22044 tablet_bootstrap.cc:654] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:42.762722 22043 tablet_bootstrap.cc:654] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: Neither blocks nor log segments found. Creating new log.
I20260620 14:55:42.763370 22044 log.cc:824] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:42.763419 22043 log.cc:824] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:42.765480 22043 tablet_bootstrap.cc:492] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: No bootstrap required, opened a new log
I20260620 14:55:42.765489 22044 tablet_bootstrap.cc:492] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: No bootstrap required, opened a new log
I20260620 14:55:42.765691 22044 ts_tablet_manager.cc:1397] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260620 14:55:42.765686 22043 ts_tablet_manager.cc:1397] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:55:42.769191 22044 raft_consensus.cc:348] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:42.769212 22043 raft_consensus.cc:348] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:42.769510 22043 raft_consensus.cc:374] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:42.769554 22044 raft_consensus.cc:374] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:55:42.769615 22043 raft_consensus.cc:729] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dbb901be3cf4960be2cdaafad6262e5, State: Initialized, Role: FOLLOWER
I20260620 14:55:42.769658 22044 raft_consensus.cc:729] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230f2d6748bd4a41a2731326c02b0f89, State: Initialized, Role: FOLLOWER
I20260620 14:55:42.770112 22043 consensus_queue.cc:260] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:42.770200 22044 consensus_queue.cc:260] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:42.770857 21898 heartbeater.cc:503] Master 127.7.24.254:32837 was elected leader, sending a full tablet report...
I20260620 14:55:42.770982 22030 heartbeater.cc:503] Master 127.7.24.254:32837 was elected leader, sending a full tablet report...
I20260620 14:55:42.771260 22043 ts_tablet_manager.cc:1428] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 14:55:42.771313 22044 ts_tablet_manager.cc:1428] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
W20260620 14:55:42.806737 21761 tablet.cc:2369] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 14:55:42.954288 22031 tablet.cc:2369] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 14:55:42.992867 21899 tablet.cc:2369] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:55:43.005453 21766 raft_consensus.cc:482] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:55:43.005759 21766 raft_consensus.cc:504] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:43.006947 21766 leader_election.cc:288] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437), 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419)
I20260620 14:55:43.011706 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.007380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:33923 (local address 127.7.24.195:36437)
0620 14:55:43.007546 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:43.007551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:43.007821 (+ 270us) server_negotiation.cc:407] Connection header received
0620 14:55:43.008034 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:43.008037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:43.008093 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:43.008192 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:43.008550 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:43.009391 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:43.010616 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:43.011026 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:43.011124 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:43.011192 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:43.011513 (+ 321us) server_negotiation.cc:299] Negotiation successful
0620 14:55:43.011588 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260620 14:55:43.011706 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.007380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:35309 (local address 127.7.24.194:33419)
0620 14:55:43.007580 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:43.007589 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:43.007838 (+ 249us) server_negotiation.cc:407] Connection header received
0620 14:55:43.008034 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:43.008037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:43.008093 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:43.008193 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:55:43.008570 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:43.009360 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:43.010613 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:43.011110 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:43.011239 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:43.011291 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:43.011474 (+ 183us) server_negotiation.cc:299] Negotiation successful
0620 14:55:43.011559 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260620 14:55:43.011725 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.007253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:33419 (local address 127.7.24.193:35309)
0620 14:55:43.007759 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.007772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:43.007894 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:43.008225 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:43.008228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:43.008269 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:43.008457 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:43.008466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:43.009525 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:43.009528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:43.010353 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:43.010362 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:43.011251 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:43.011254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:43.011347 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:43.011364 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 14:55:43.011431 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 14:55:43.011534 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":60,"spinlock_wait_cycles":6016,"thread_start_us":61,"threads_started":1}
I20260620 14:55:43.011758 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.007253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:36437 (local address 127.7.24.193:33923)
0620 14:55:43.007704 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.007721 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:43.007887 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:43.008225 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:43.008228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:43.008267 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:55:43.008400 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:43.008410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:43.009549 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:43.009553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:43.010396 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:43.010408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:43.011167 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:43.011174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:43.011277 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:43.011375 (+ 98us) client_negotiation.cc:769] Sending connection context
0620 14:55:43.011485 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 14:55:43.011560 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":34,"spinlock_wait_cycles":27264,"thread_start_us":119,"threads_started":1}
I20260620 14:55:43.012377 21852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8c70bf5474462b93a4bc52f1267708" candidate_uuid: "d281af84fa3948838fb0c11ae1729788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dbb901be3cf4960be2cdaafad6262e5" is_pre_election: true
I20260620 14:55:43.012471 21984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8c70bf5474462b93a4bc52f1267708" candidate_uuid: "d281af84fa3948838fb0c11ae1729788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "230f2d6748bd4a41a2731326c02b0f89" is_pre_election: true
I20260620 14:55:43.012730 21852 raft_consensus.cc:2454] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d281af84fa3948838fb0c11ae1729788 in term 0.
I20260620 14:55:43.012776 21984 raft_consensus.cc:2454] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d281af84fa3948838fb0c11ae1729788 in term 0.
I20260620 14:55:43.013247 21647 leader_election.cc:302] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dbb901be3cf4960be2cdaafad6262e5, d281af84fa3948838fb0c11ae1729788; no voters:
I20260620 14:55:43.013545 21766 raft_consensus.cc:2781] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:55:43.013633 21766 raft_consensus.cc:482] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:55:43.013676 21766 raft_consensus.cc:3037] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:43.014752 21766 raft_consensus.cc:504] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:43.015098 21766 leader_election.cc:288] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 1 election: Requested vote from peers 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437), 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419)
I20260620 14:55:43.015506 21984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8c70bf5474462b93a4bc52f1267708" candidate_uuid: "d281af84fa3948838fb0c11ae1729788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "230f2d6748bd4a41a2731326c02b0f89"
I20260620 14:55:43.015654 21984 raft_consensus.cc:3037] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:43.016749 21984 raft_consensus.cc:2454] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d281af84fa3948838fb0c11ae1729788 in term 1.
I20260620 14:55:43.017046 21852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8c70bf5474462b93a4bc52f1267708" candidate_uuid: "d281af84fa3948838fb0c11ae1729788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dbb901be3cf4960be2cdaafad6262e5"
I20260620 14:55:43.017067 21648 leader_election.cc:302] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230f2d6748bd4a41a2731326c02b0f89, d281af84fa3948838fb0c11ae1729788; no voters:
I20260620 14:55:43.017163 21852 raft_consensus.cc:3037] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:55:43.017270 21766 raft_consensus.cc:2781] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:55:43.017477 21766 raft_consensus.cc:686] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 LEADER]: Becoming Leader. State: Replica: d281af84fa3948838fb0c11ae1729788, State: Running, Role: LEADER
I20260620 14:55:43.017761 21766 consensus_queue.cc:237] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:43.018515 21852 raft_consensus.cc:2454] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d281af84fa3948838fb0c11ae1729788 in term 1.
I20260620 14:55:43.021160 21572 catalog_manager.cc:5495] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 reported cstate change: term changed from 0 to 1, leader changed from <none> to d281af84fa3948838fb0c11ae1729788 (127.7.24.193). New cstate: current_term: 1 leader_uuid: "d281af84fa3948838fb0c11ae1729788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } health_report { overall_health: HEALTHY } } }
I20260620 14:55:43.053848 22054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.050075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34516 (local address 127.7.24.193:33307)
0620 14:55:43.050407 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:43.050415 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:43.050445 (+ 30us) server_negotiation.cc:407] Connection header received
0620 14:55:43.050581 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:43.050584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:43.050640 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:43.050743 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:55:43.051023 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:43.051558 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:43.052379 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:43.052624 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:43.052705 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:43.052959 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:55:43.053386 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:55:43.053476 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:43.053576 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 14:55:43.053624 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":133,"threads_started":1}
I20260620 14:55:43.057905 21984 raft_consensus.cc:1261] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89 [term 1 FOLLOWER]: Refusing update from remote peer d281af84fa3948838fb0c11ae1729788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:55:43.058025 21852 raft_consensus.cc:1261] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5 [term 1 FOLLOWER]: Refusing update from remote peer d281af84fa3948838fb0c11ae1729788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:55:43.058790 22051 consensus_queue.cc:1035] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:55:43.059013 21766 consensus_queue.cc:1035] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:55:43.068137 22057 mvcc.cc:205] Tried to move back new op lower bound from 7298938237158490112 to 7298938237005115392. Current Snapshot: MvccSnapshot[applied={T|T < 7298938237158490112 or (T in {7298938237158490112})}]
I20260620 14:55:43.074116 21572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:55:43.074304 21572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:55:43.076854 21572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Sending DeleteTablet for 3 replicas of tablet 7d8c70bf5474462b93a4bc52f1267708
I20260620 14:55:43.077627 21964 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d8c70bf5474462b93a4bc52f1267708 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:55:43 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:57674
I20260620 14:55:43.077847 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 21610
I20260620 14:55:43.078176 21832 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d8c70bf5474462b93a4bc52f1267708 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:55:43 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:57762
I20260620 14:55:43.078912 22066 tablet_replica.cc:330] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: stopping tablet replica
I20260620 14:55:43.079238 22066 raft_consensus.cc:2229] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:55:43.079567 22066 raft_consensus.cc:2258] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:55:43.080003 22065 tablet_replica.cc:330] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: stopping tablet replica
I20260620 14:55:43.080303 22065 raft_consensus.cc:2229] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:55:43.080603 22065 raft_consensus.cc:2258] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:55:43.081373 22066 ts_tablet_manager.cc:1905] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:55:43.081943 22065 ts_tablet_manager.cc:1905] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:55:43.084460 22066 ts_tablet_manager.cc:1918] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:55:43.084579 22066 log.cc:1196] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/7d8c70bf5474462b93a4bc52f1267708
I20260620 14:55:43.084759 22065 ts_tablet_manager.cc:1918] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:55:43.084870 22065 log.cc:1196] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/7d8c70bf5474462b93a4bc52f1267708
I20260620 14:55:43.085029 22066 ts_tablet_manager.cc:1939] T 7d8c70bf5474462b93a4bc52f1267708 P 0dbb901be3cf4960be2cdaafad6262e5: Deleting consensus metadata
I20260620 14:55:43.085249 22065 ts_tablet_manager.cc:1939] T 7d8c70bf5474462b93a4bc52f1267708 P 230f2d6748bd4a41a2731326c02b0f89: Deleting consensus metadata
W20260620 14:55:43.086256 21557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:33307 (error 108)
I20260620 14:55:43.086305 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 21764
I20260620 14:55:43.088521 21767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.088234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33307 (local address 127.0.0.1:34524)
0620 14:55:43.088372 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.088446 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:33307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260620 14:55:43.088672 21557 catalog_manager.cc:4584] TS d281af84fa3948838fb0c11ae1729788 (127.7.24.193:33307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d8c70bf5474462b93a4bc52f1267708: Network error: Client connection negotiation failed: client connection to 127.7.24.193:33307: connect: Connection refused (error 111)
I20260620 14:55:43.090675 21558 catalog_manager.cc:4841] TS 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437): tablet 7d8c70bf5474462b93a4bc52f1267708 (table test-table [id=0ff693def0624345b5f360ed2960e61f]) successfully deleted
I20260620 14:55:43.092646 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 21901
I20260620 14:55:43.093657 21767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.092801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:33419 (local address 127.0.0.1:57774)
0620 14:55:43.093535 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.093588 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.194:33419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":666}
I20260620 14:55:43.099653 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 21539
I20260620 14:55:43.106171 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:32837
--webserver_interface=127.7.24.254
--webserver_port=36655
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:32837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:55:43.217175 22068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:43.217446 22068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:43.217489 22068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:43.221472 22068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:55:43.221539 22068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:43.221558 22068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:43.221575 22068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:55:43.221591 22068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:55:43.227146 22068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:32837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:32837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=36655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:43.228225 22068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:43.229256 22068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:43.237686 22073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:43.237715 22076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:43.237728 22074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:43.238152 22068 server_base.cc:1034] running on GCE node
I20260620 14:55:43.238775 22068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:43.239842 22068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:43.241019 22068 hybrid_clock.cc:648] HybridClock initialized: now 1781967343241002 us; error 25 us; skew 500 ppm
Jun 20 14:55:43 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967343, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:43.243777 22068 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:55:43.244670 22068 webserver.cc:460] Webserver started at http://127.7.24.254:36655/ using document root <none> and password file <none>
I20260620 14:55:43.245222 22068 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:43.245316 22068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:43.249083 22068 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:55:43.251318 22083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:43.252592 22068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:55:43.252745 22068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "7a44cc72cc234d6a8d4899044246b272"
format_stamp: "Formatted at 2026-06-20 14:55:40 on dist-test-slave-sms9"
server_key: "fe713ae040ce30f99d61a659f88c7016"
server_key_iv: "1b04095d3d9f0cb2c7e3bb4530069c0f"
server_key_version: "encryptionkey@0"
I20260620 14:55:43.253304 22068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:43.277022 22068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:43.280306 22068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:43.280536 22068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:43.287940 22068 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:32837
I20260620 14:55:43.288233 22135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:32837 every 8 connection(s)
I20260620 14:55:43.289047 22068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:55:43.292797 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 22068
I20260620 14:55:43.293249 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:33307
--local_ip_for_outbound_sockets=127.7.24.193
--tserver_master_addrs=127.7.24.254:32837
--webserver_port=37875
--webserver_interface=127.7.24.193
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:55:43.293387 22136 sys_catalog.cc:263] Verifying existing consensus state
I20260620 14:55:43.295065 22136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Bootstrap starting.
I20260620 14:55:43.312510 22136 log.cc:824] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:43.324023 22136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:55:43.324424 22136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272: Bootstrap complete.
I20260620 14:55:43.328540 22136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:43.328949 22136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a44cc72cc234d6a8d4899044246b272, State: Initialized, Role: FOLLOWER
I20260620 14:55:43.329445 22136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:43.329589 22136 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:55:43.329679 22136 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:55:43.329802 22136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 2 FOLLOWER]: Advancing to term 3
I20260620 14:55:43.331760 22136 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:43.332098 22136 leader_election.cc:302] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a44cc72cc234d6a8d4899044246b272; no voters:
I20260620 14:55:43.332448 22136 leader_election.cc:288] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 14:55:43.332609 22140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 14:55:43.332842 22140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [term 3 LEADER]: Becoming Leader. State: Replica: 7a44cc72cc234d6a8d4899044246b272, State: Running, Role: LEADER
I20260620 14:55:43.333204 22140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } }
I20260620 14:55:43.333642 22136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:55:43.334640 22141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7a44cc72cc234d6a8d4899044246b272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } } }
I20260620 14:55:43.334769 22141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:43.335121 22142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a44cc72cc234d6a8d4899044246b272. Latest consensus state: current_term: 3 leader_uuid: "7a44cc72cc234d6a8d4899044246b272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a44cc72cc234d6a8d4899044246b272" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 32837 } } }
I20260620 14:55:43.335282 22142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a44cc72cc234d6a8d4899044246b272 [sys.catalog]: This master's current role is: LEADER
I20260620 14:55:43.335639 22149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:55:43.339847 22149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:55:43.340384 22149 catalog_manager.cc:1259] Loaded cluster ID: f40cd1436dc849c3842dc7d456cd6213
I20260620 14:55:43.340462 22149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260620 14:55:43.341288 22149 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa82c89d9ad google::LogMessage::Fail() at ??:0
@ 0x7fa82c8a088a google::LogMessage::SendToLog() at ??:0
@ 0x7fa82c89d4de google::LogMessage::Flush() at ??:0
@ 0x7fa82c89ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa8371dcfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fa8371ddd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa8371da579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fa83721e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa82eaef3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa82eaefc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fa82eaf13f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa82eae0512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa8301246db start_thread at ??:0
@ 0x7fa82b39b61f clone at ??:0
W20260620 14:55:43.423776 22138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:43.424096 22138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:43.424170 22138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:43.428220 22138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:43.428297 22138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:43.428402 22138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:55:43.433727 22138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:33307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=37875
--tserver_master_addrs=127.7.24.254:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:43.434861 22138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:43.435925 22138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:43.445508 22178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:43.445505 22177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:43.445721 22138 server_base.cc:1034] running on GCE node
W20260620 14:55:43.445811 22180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:43.446233 22138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:43.446938 22138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:43.448118 22138 hybrid_clock.cc:648] HybridClock initialized: now 1781967343448105 us; error 37 us; skew 500 ppm
Jun 20 14:55:43 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967343, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:43.450896 22138 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:55:43.451866 22138 webserver.cc:460] Webserver started at http://127.7.24.193:37875/ using document root <none> and password file <none>
I20260620 14:55:43.452425 22138 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:43.452486 22138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:43.456630 22138 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:55:43.458890 22187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:43.460249 22138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:55:43.460386 22138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "d281af84fa3948838fb0c11ae1729788"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "7e888961baded3022b4df79240d27d47"
server_key_iv: "160c1162677a846736c7cf0fd9ee798c"
server_key_version: "encryptionkey@0"
I20260620 14:55:43.460937 22138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:43.480363 22138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:43.483621 22138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:43.483832 22138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:43.484416 22138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:43.485535 22194 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 14:55:43.490823 22138 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 14:55:43.490897 22138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260620 14:55:43.490986 22138 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 14:55:43.492591 22138 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 14:55:43.492661 22138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260620 14:55:43.492934 22194 tablet_bootstrap.cc:492] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: Bootstrap starting.
I20260620 14:55:43.503854 22138 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:33307
I20260620 14:55:43.505229 22138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:55:43.505640 22301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:33307 every 8 connection(s)
I20260620 14:55:43.511298 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 22138
I20260620 14:55:43.511747 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:33419
--local_ip_for_outbound_sockets=127.7.24.194
--tserver_master_addrs=127.7.24.254:32837
--webserver_port=38583
--webserver_interface=127.7.24.194
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:55:43.516167 22194 log.cc:824] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: Log is configured to *not* fsync() on all Append() calls
I20260620 14:55:43.525883 22194 tablet_bootstrap.cc:492] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:55:43.526396 22194 tablet_bootstrap.cc:492] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: Bootstrap complete.
I20260620 14:55:43.527657 22194 ts_tablet_manager.cc:1397] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: Time spent bootstrapping tablet: real 0.035s user 0.022s sys 0.010s
I20260620 14:55:43.531008 22194 raft_consensus.cc:348] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:43.531528 22194 raft_consensus.cc:729] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d281af84fa3948838fb0c11ae1729788, State: Initialized, Role: FOLLOWER
I20260620 14:55:43.532198 22194 consensus_queue.cc:260] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:43.533221 22194 ts_tablet_manager.cc:1428] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260620 14:55:43.585140 22304 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:55:43.507610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:53897)
0620 14:55:43.508103 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.508156 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:43.508967 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:43.584174 (+ 75207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":160,"threads_started":1}
W20260620 14:55:43.585817 22302 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:32837 (0 consecutive failures): Network error: Failed to ping master at 127.7.24.254:32837: Client connection negotiation failed: client connection to 127.7.24.254:32837: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 14:55:43.586555 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.586272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:44169)
0620 14:55:43.586406 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.586495 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 14:55:43.587193 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.586955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:54035)
0620 14:55:43.587080 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.587138 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 14:55:43.587402 22302 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 14:55:43.623998 22305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:43.624238 22305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:43.624280 22305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:43.628217 22305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:43.628271 22305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:43.628329 22305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:55:43.633342 22305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:33419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=38583
--tserver_master_addrs=127.7.24.254:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:43.634267 22305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:43.635264 22305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:43.644920 22313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:43.644914 22312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:43.644915 22315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:43.645174 22305 server_base.cc:1034] running on GCE node
I20260620 14:55:43.645727 22305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:43.646284 22305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:43.647477 22305 hybrid_clock.cc:648] HybridClock initialized: now 1781967343647443 us; error 47 us; skew 500 ppm
Jun 20 14:55:43 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967343, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:43.650179 22305 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:55:43.651067 22305 webserver.cc:460] Webserver started at http://127.7.24.194:38583/ using document root <none> and password file <none>
I20260620 14:55:43.651567 22305 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:43.651624 22305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:43.655375 22305 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:55:43.657451 22322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:43.658747 22305 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:55:43.658839 22305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "0dbb901be3cf4960be2cdaafad6262e5"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "e7272bcec462c852b1085c95274ecc19"
server_key_iv: "76a2a6e3a40bed961ec47ce383cdcd2b"
server_key_version: "encryptionkey@0"
I20260620 14:55:43.659375 22305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:43.691512 22305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:43.694892 22305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:43.695103 22305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:43.695719 22305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:43.696556 22305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:43.696604 22305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:43.696643 22305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:43.696659 22305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:43.706017 22305 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:33419
I20260620 14:55:43.706041 22435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:33419 every 8 connection(s)
I20260620 14:55:43.707123 22305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:55:43.708736 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 22305
I20260620 14:55:43.709192 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:36437
--local_ip_for_outbound_sockets=127.7.24.195
--tserver_master_addrs=127.7.24.254:32837
--webserver_port=36821
--webserver_interface=127.7.24.195
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:55:43.711735 22438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.709601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:35485)
0620 14:55:43.710415 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.710663 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":149,"threads_started":1}
W20260620 14:55:43.712258 22436 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:32837 (0 consecutive failures): Network error: Failed to ping master at 127.7.24.254:32837: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
I20260620 14:55:43.712949 22438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.712706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:48783)
0620 14:55:43.712827 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.712882 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260620 14:55:43.713608 22438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.713365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:33955)
0620 14:55:43.713469 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.713518 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260620 14:55:43.713774 22436 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 14:55:43.820950 22439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:55:43.821233 22439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:55:43.821307 22439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:55:43.825466 22439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:55:43.825555 22439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:55:43.825645 22439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:55:43.830971 22439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:36437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=36821
--tserver_master_addrs=127.7.24.254:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:55:43.832026 22439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:55:43.833041 22439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:55:43.843801 22445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:43.843823 22447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:55:43.843806 22444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:55:43.844041 22439 server_base.cc:1034] running on GCE node
I20260620 14:55:43.844512 22439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:55:43.845293 22439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:55:43.846510 22439 hybrid_clock.cc:648] HybridClock initialized: now 1781967343846450 us; error 26 us; skew 500 ppm
Jun 20 14:55:43 dist-test-slave-sms9 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967343, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:55:43.849179 22439 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:55:43.849982 22439 webserver.cc:460] Webserver started at http://127.7.24.195:36821/ using document root <none> and password file <none>
I20260620 14:55:43.850574 22439 fs_manager.cc:359] Metadata directory not provided
I20260620 14:55:43.850646 22439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:55:43.854414 22439 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:55:43.856592 22454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:43.857817 22439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:55:43.857944 22439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "230f2d6748bd4a41a2731326c02b0f89"
format_stamp: "Formatted at 2026-06-20 14:55:41 on dist-test-slave-sms9"
server_key: "46858317258a5ed84fb58688e574dcd9"
server_key_iv: "99daaef6a8a9de50c0e8c2808e28ab37"
server_key_version: "encryptionkey@0"
I20260620 14:55:43.858546 22439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:55:43.871726 22439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:55:43.875000 22439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:55:43.875159 22439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:55:43.875686 22439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:55:43.876565 22439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:55:43.876612 22439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:43.876650 22439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:55:43.876667 22439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:55:43.885699 22439 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:36437
I20260620 14:55:43.885725 22567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:36437 every 8 connection(s)
I20260620 14:55:43.886791 22439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:55:43.890668 22570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.889004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:59859)
0620 14:55:43.889715 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.889844 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":109,"threads_started":1}
W20260620 14:55:43.891216 22568 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:32837 (0 consecutive failures): Network error: Failed to ping master at 127.7.24.254:32837: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
I20260620 14:55:43.891850 22570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.891600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:50213)
0620 14:55:43.891722 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.891775 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260620 14:55:43.892563 22570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:43.892274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:32991)
0620 14:55:43.892450 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:43.892510 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260620 14:55:43.892735 22568 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 14:55:43.896623 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 22439
I20260620 14:55:44.588469 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:44.588018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:39169)
0620 14:55:44.588292 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:44.588366 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":77,"threads_started":1}
I20260620 14:55:44.714778 22573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:44.714317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:52817)
0620 14:55:44.714618 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:44.714674 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":90,"threads_started":1}
I20260620 14:55:44.893817 22574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:44.893367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:55727)
0620 14:55:44.893655 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:44.893715 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":76,"threads_started":1}
I20260620 14:55:45.332572 22575 raft_consensus.cc:482] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:55:45.332773 22575 raft_consensus.cc:504] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:45.334129 22575 leader_election.cc:288] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437), 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419)
Jun 20 14:55:45 dist-test-slave-sms9 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967343, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.195@KRBTEST.COM
Jun 20 14:55:45 dist-test-slave-sms9 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967343, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.194@KRBTEST.COM
I20260620 14:55:45.348822 22576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:45.334284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:36437 (local address 127.7.24.193:41193)
0620 14:55:45.334658 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:45.334684 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:45.334795 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:45.336445 (+ 1650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:45.336448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:45.337015 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:45.337495 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:45.337507 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:45.338945 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:45.338948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:45.339873 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:45.339883 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:45.340639 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:45.340643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:45.340861 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:45.341601 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:45.341621 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:45.343365 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:45.345962 (+ 2597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:45.345972 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:45.345989 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:45.346319 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:45.346807 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:45.346810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:45.346811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:45.347024 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:45.347827 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:45.347834 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:45.348073 (+ 239us) client_negotiation.cc:769] Sending connection context
0620 14:55:45.348259 (+ 186us) client_negotiation.cc:240] Negotiation successful
0620 14:55:45.348626 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":160,"spinlock_wait_cycles":4608,"thread_start_us":79,"threads_started":1}
I20260620 14:55:45.348845 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:45.334608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:50031 (local address 127.7.24.194:33419)
0620 14:55:45.335014 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:45.335024 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:45.335054 (+ 30us) server_negotiation.cc:407] Connection header received
0620 14:55:45.335710 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:45.335716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:45.336064 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:45.336380 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:45.337683 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:45.338715 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:45.340041 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:45.340491 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:45.340659 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:45.343986 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:45.343991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:45.344001 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:45.344029 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:45.346261 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:45.346927 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:45.346933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:45.346938 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:45.347068 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:45.347402 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:45.347405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:45.347406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:45.347778 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:45.347938 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:45.348383 (+ 445us) server_negotiation.cc:299] Negotiation successful
0620 14:55:45.348648 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":90,"threads_started":1}
I20260620 14:55:45.348902 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:45.334509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:41193 (local address 127.7.24.195:36437)
0620 14:55:45.334950 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0620 14:55:45.334979 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0620 14:55:45.335024 (+ 45us) server_negotiation.cc:407] Connection header received
0620 14:55:45.335736 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:55:45.335741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:55:45.336068 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:55:45.336381 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:55:45.337670 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:45.338809 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:45.339998 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:55:45.340493 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:55:45.340677 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:45.343659 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:55:45.343666 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:55:45.343685 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:55:45.343731 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:55:45.345836 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:45.346452 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:45.346490 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:45.346496 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:45.346678 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:55:45.347162 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:55:45.347166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:55:45.347168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:55:45.347700 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:55:45.347813 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 14:55:45.348385 (+ 572us) server_negotiation.cc:299] Negotiation successful
0620 14:55:45.348692 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":79,"threads_started":1}
I20260620 14:55:45.348949 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:45.334378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:33419 (local address 127.7.24.193:50031)
0620 14:55:45.334798 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:45.334813 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:55:45.334918 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:55:45.336442 (+ 1524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:55:45.336448 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:55:45.337004 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:55:45.337491 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:45.337507 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:45.338855 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:45.338862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:45.339875 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:55:45.339885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:55:45.340623 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:55:45.340626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:55:45.340836 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:55:45.341646 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:55:45.341660 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:55:45.343789 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:55:45.346409 (+ 2620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:45.346499 (+ 90us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:45.346501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:45.346797 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:45.347168 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:55:45.347171 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:55:45.347173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:55:45.347293 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:55:45.347922 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:55:45.347925 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:55:45.348071 (+ 146us) client_negotiation.cc:769] Sending connection context
0620 14:55:45.348259 (+ 188us) client_negotiation.cc:240] Negotiation successful
0620 14:55:45.348721 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":202,"spinlock_wait_cycles":26624,"thread_start_us":78,"threads_started":1}
W20260620 14:55:45.351536 22189 leader_election.cc:341] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419): Not found: Tablet not found: 7d8c70bf5474462b93a4bc52f1267708
W20260620 14:55:45.351707 22190 leader_election.cc:341] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437): Not found: Tablet not found: 7d8c70bf5474462b93a4bc52f1267708
I20260620 14:55:45.351799 22190 leader_election.cc:302] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d281af84fa3948838fb0c11ae1729788; no voters: 0dbb901be3cf4960be2cdaafad6262e5, 230f2d6748bd4a41a2731326c02b0f89
I20260620 14:55:45.352069 22575 raft_consensus.cc:2726] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 14:55:45.589560 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:45.589263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:48731)
0620 14:55:45.589404 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:45.589468 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260620 14:55:45.716133 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:45.715583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:40195)
0620 14:55:45.715853 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:45.715915 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":98,"threads_started":1}
I20260620 14:55:45.895066 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:45.894662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:57621)
0620 14:55:45.894928 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:45.894979 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":103,"threads_started":1}
I20260620 14:55:46.590847 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:46.590360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:34551)
0620 14:55:46.590677 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:46.590739 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":109,"threads_started":1}
I20260620 14:55:46.717432 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:46.716996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:51059)
0620 14:55:46.717269 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:46.717333 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":80,"threads_started":1}
I20260620 14:55:46.896183 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:46.895749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:51843)
0620 14:55:46.896007 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:46.896079 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":80,"threads_started":1}
I20260620 14:55:46.913192 22585 raft_consensus.cc:482] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:55:46.913347 22585 raft_consensus.cc:504] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:46.913875 22585 leader_election.cc:288] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437), 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419)
W20260620 14:55:46.914685 22190 leader_election.cc:341] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437): Not found: Tablet not found: 7d8c70bf5474462b93a4bc52f1267708
W20260620 14:55:46.914821 22189 leader_election.cc:341] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419): Not found: Tablet not found: 7d8c70bf5474462b93a4bc52f1267708
I20260620 14:55:46.914917 22189 leader_election.cc:302] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d281af84fa3948838fb0c11ae1729788; no voters: 0dbb901be3cf4960be2cdaafad6262e5, 230f2d6748bd4a41a2731326c02b0f89
I20260620 14:55:46.915055 22585 raft_consensus.cc:2726] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 14:55:47.592095 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:47.591681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:57453)
0620 14:55:47.591933 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:47.591992 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":77,"threads_started":1}
I20260620 14:55:47.718863 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:47.718311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:42209)
0620 14:55:47.718662 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:47.718731 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":84,"threads_started":1}
I20260620 14:55:47.897326 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:47.896907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:40483)
0620 14:55:47.897180 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:47.897231 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
I20260620 14:55:48.591162 22589 raft_consensus.cc:482] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:55:48.591301 22589 raft_consensus.cc:504] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:48.591825 22589 leader_election.cc:288] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437), 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419)
W20260620 14:55:48.592728 22190 leader_election.cc:341] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437): Not found: Tablet not found: 7d8c70bf5474462b93a4bc52f1267708
W20260620 14:55:48.592872 22189 leader_election.cc:341] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419): Not found: Tablet not found: 7d8c70bf5474462b93a4bc52f1267708
I20260620 14:55:48.592993 22189 leader_election.cc:302] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d281af84fa3948838fb0c11ae1729788; no voters: 0dbb901be3cf4960be2cdaafad6262e5, 230f2d6748bd4a41a2731326c02b0f89
I20260620 14:55:48.593160 22589 raft_consensus.cc:2726] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 14:55:48.593449 22590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:48.593024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:60171)
0620 14:55:48.593301 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:48.593371 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":69,"threads_started":1}
I20260620 14:55:48.720181 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:48.719760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:46045)
0620 14:55:48.720012 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:48.720073 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":75,"threads_started":1}
I20260620 14:55:48.898571 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:48.898095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:59409)
0620 14:55:48.898356 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:48.898413 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":84,"threads_started":1}
I20260620 14:55:49.594553 22593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:49.594114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:38185)
0620 14:55:49.594375 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:49.594429 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":94,"threads_started":1}
I20260620 14:55:49.721506 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:49.721013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:37641)
0620 14:55:49.721294 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:49.721406 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":79,"threads_started":1}
I20260620 14:55:49.899763 22595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:49.899333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:43897)
0620 14:55:49.899619 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:49.899672 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":108,"threads_started":1}
I20260620 14:55:50.551734 22596 raft_consensus.cc:482] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:55:50.551872 22596 raft_consensus.cc:504] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:50.552399 22596 leader_election.cc:288] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437), 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419)
W20260620 14:55:50.553303 22189 leader_election.cc:341] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419): Not found: Tablet not found: 7d8c70bf5474462b93a4bc52f1267708
W20260620 14:55:50.553493 22190 leader_election.cc:341] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437): Not found: Tablet not found: 7d8c70bf5474462b93a4bc52f1267708
I20260620 14:55:50.553576 22190 leader_election.cc:302] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d281af84fa3948838fb0c11ae1729788; no voters: 0dbb901be3cf4960be2cdaafad6262e5, 230f2d6748bd4a41a2731326c02b0f89
I20260620 14:55:50.553732 22596 raft_consensus.cc:2726] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 14:55:50.595832 22597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:50.595411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:59785)
0620 14:55:50.595680 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:50.595736 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260620 14:55:50.722899 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:50.722428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:41755)
0620 14:55:50.722732 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:50.722791 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":70,"threads_started":1}
I20260620 14:55:50.901142 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:50.900650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:36359)
0620 14:55:50.900977 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:50.901042 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":78,"threads_started":1}
I20260620 14:55:51.597188 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:51.596748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:56789)
0620 14:55:51.597026 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:51.597090 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":80,"threads_started":1}
I20260620 14:55:51.724215 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:51.723689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:37331)
0620 14:55:51.724062 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:51.724117 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":210,"threads_started":1}
I20260620 14:55:51.902451 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:51.901965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:53151)
0620 14:55:51.902234 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:51.902295 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":80,"threads_started":1}
I20260620 14:55:52.598639 22603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:52.598080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:34493)
0620 14:55:52.598367 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:52.598428 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":92,"threads_started":1}
I20260620 14:55:52.725512 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:52.725100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:39691)
0620 14:55:52.725354 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:52.725417 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
I20260620 14:55:52.903883 22605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:52.903422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:38819)
0620 14:55:52.903716 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:52.903777 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":111,"threads_started":1}
I20260620 14:55:53.599921 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:53.599516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:58677)
0620 14:55:53.599757 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:53.599811 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
I20260620 14:55:53.727160 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:53.726681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:54683)
0620 14:55:53.726992 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:53.727059 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":76,"threads_started":1}
I20260620 14:55:53.905339 22608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:53.904825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:53469)
0620 14:55:53.905158 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:53.905223 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
I20260620 14:55:54.601277 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:54.600840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:52015)
0620 14:55:54.601116 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:54.601179 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":82,"threads_started":1}
I20260620 14:55:54.728619 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:54.728130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:53559)
0620 14:55:54.728451 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:54.728518 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":79,"threads_started":1}
I20260620 14:55:54.906828 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:54.906324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:58605)
0620 14:55:54.906665 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:54.906728 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":77,"threads_started":1}
I20260620 14:55:55.602646 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:55.602203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:51303)
0620 14:55:55.602485 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:55.602544 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":80,"threads_started":1}
I20260620 14:55:55.729995 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:55.729537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:44903)
0620 14:55:55.729820 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:55.729886 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
I20260620 14:55:55.908046 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:55.907633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:50823)
0620 14:55:55.907898 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:55.907951 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":85,"threads_started":1}
I20260620 14:55:56.603766 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:56.603373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:49317)
0620 14:55:56.603619 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:56.603671 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":83,"threads_started":1}
I20260620 14:55:56.731458 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:56.730959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:34117)
0620 14:55:56.731277 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:56.731344 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":78,"threads_started":1}
I20260620 14:55:56.909296 22617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:56.908861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:44905)
0620 14:55:56.909137 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:56.909198 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":89,"threads_started":1}
I20260620 14:55:57.343580 22618 raft_consensus.cc:482] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:55:57.343710 22618 raft_consensus.cc:504] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "230f2d6748bd4a41a2731326c02b0f89" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 36437 } } peers { permanent_uuid: "0dbb901be3cf4960be2cdaafad6262e5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 33419 } } peers { permanent_uuid: "d281af84fa3948838fb0c11ae1729788" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33307 } }
I20260620 14:55:57.344192 22618 leader_election.cc:288] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437), 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419)
W20260620 14:55:57.345229 22190 leader_election.cc:341] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 230f2d6748bd4a41a2731326c02b0f89 (127.7.24.195:36437): Not found: Tablet not found: 7d8c70bf5474462b93a4bc52f1267708
W20260620 14:55:57.345341 22189 leader_election.cc:341] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0dbb901be3cf4960be2cdaafad6262e5 (127.7.24.194:33419): Not found: Tablet not found: 7d8c70bf5474462b93a4bc52f1267708
I20260620 14:55:57.345400 22189 leader_election.cc:302] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d281af84fa3948838fb0c11ae1729788; no voters: 0dbb901be3cf4960be2cdaafad6262e5, 230f2d6748bd4a41a2731326c02b0f89
I20260620 14:55:57.345600 22618 raft_consensus.cc:2726] T 7d8c70bf5474462b93a4bc52f1267708 P d281af84fa3948838fb0c11ae1729788 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 14:55:57.605072 22619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:57.604640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:54851)
0620 14:55:57.604919 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:57.604973 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":88,"threads_started":1}
I20260620 14:55:57.732848 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:57.732337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:52787)
0620 14:55:57.732677 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:57.732742 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":134,"threads_started":1}
I20260620 14:55:57.910672 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:57.910179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.195:51153)
0620 14:55:57.910511 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:57.910572 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":94,"threads_started":1}
I20260620 14:55:58.606509 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:58.606030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.193:40831)
0620 14:55:58.606330 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:58.606386 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":100,"threads_started":1}
I20260620 14:55:58.734207 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:55:58.733784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:32837 (local address 127.7.24.194:45701)
0620 14:55:58.734055 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0620 14:55:58.734111 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":84,"threads_started":1}
I20260620 14:55:58.900051 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 22138
I20260620 14:55:58.905891 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 22305
I20260620 14:55:58.911418 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 22439
2026-06-20T14:55:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19320 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:55:58 dist-test-slave-sms9 krb5kdc[22626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:55:58 dist-test-slave-sms9 krb5kdc[22626](info): set up 2 sockets
Jun 20 14:55:58 dist-test-slave-sms9 krb5kdc[22626](info): commencing operation
krb5kdc: starting...
W20260620 14:55:59.994828 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:56:00 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:56:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:56:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:56:00.121749 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40807
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:56:00.235747 22639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:00.236110 22639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:00.236189 22639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:00.240338 22639 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 14:56:00.240438 22639 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 14:56:00.240475 22639 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 14:56:00.240506 22639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:56:00.240550 22639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:00.240589 22639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:00.240626 22639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:56:00.240659 22639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:56:00.246209 22639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:00.247406 22639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:00.248489 22639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:00.257942 22644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:00.257995 22647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:00.258066 22639 server_base.cc:1034] running on GCE node
W20260620 14:56:00.258002 22645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:00.259191 22639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:00.260515 22639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:00.261739 22639 hybrid_clock.cc:648] HybridClock initialized: now 1781967360261696 us; error 52 us; skew 500 ppm
Jun 20 14:56:00 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967360, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:00.264871 22639 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:56:00.265931 22639 webserver.cc:460] Webserver started at http://127.7.24.254:38021/ using document root <none> and password file <none>
I20260620 14:56:00.266573 22639 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:00.266669 22639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:00.266906 22639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:56:00.269006 22639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "85d01da39508411cb57cc674a99db034"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "ea45a129d3fc3304a1d3d847f3284267"
server_key_iv: "104e759ff72f1db40225e2bd0ede3bc7"
server_key_version: "encryptionkey@0"
I20260620 14:56:00.269590 22639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "85d01da39508411cb57cc674a99db034"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "ea45a129d3fc3304a1d3d847f3284267"
server_key_iv: "104e759ff72f1db40225e2bd0ede3bc7"
server_key_version: "encryptionkey@0"
I20260620 14:56:00.273545 22639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260620 14:56:00.276258 22654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:00.277698 22639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 14:56:00.277864 22639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "85d01da39508411cb57cc674a99db034"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "ea45a129d3fc3304a1d3d847f3284267"
server_key_iv: "104e759ff72f1db40225e2bd0ede3bc7"
server_key_version: "encryptionkey@0"
I20260620 14:56:00.277997 22639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:00.302071 22639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:00.305678 22639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:00.306035 22639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:00.313409 22639 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:40807
I20260620 14:56:00.313717 22706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:40807 every 8 connection(s)
I20260620 14:56:00.314447 22639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:56:00.317507 22707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:56:00.319304 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 22639
I20260620 14:56:00.319463 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:56:00.319772 7267 external_mini_cluster.cc:1428] Setting key c06f8b03f9d6192e8bf9f26dd902684d
I20260620 14:56:00.324934 22707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Bootstrap starting.
I20260620 14:56:00.327873 22707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Neither blocks nor log segments found. Creating new log.
I20260620 14:56:00.328693 22707 log.cc:824] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Log is configured to *not* fsync() on all Append() calls
Jun 20 14:56:00 dist-test-slave-sms9 krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:56:00.331221 22707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: No bootstrap required, opened a new log
I20260620 14:56:00.336700 22710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:00.321246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56600 (local address 127.7.24.254:40807)
0620 14:56:00.321728 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:00.321750 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:00.321797 (+ 47us) server_negotiation.cc:407] Connection header received
0620 14:56:00.322820 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:00.322846 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:00.323304 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:00.323723 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:00.324459 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:00.326056 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:00.327051 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:00.327820 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:00.328005 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:00.331019 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:00.331026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:00.331045 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:00.331078 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:00.333201 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:00.333636 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:00.333643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:00.333649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:00.333818 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:00.334134 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:00.334141 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:00.334145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:00.334796 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:00.335018 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:00.335237 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 14:56:00.335608 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":129,"threads_started":1}
I20260620 14:56:00.336695 22707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:00.337028 22707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:56:00.337091 22707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d01da39508411cb57cc674a99db034, State: Initialized, Role: FOLLOWER
I20260620 14:56:00.337575 22707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:00.337750 22707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:56:00.337859 22707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:56:00.338670 22707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:56:00.339761 22707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:00.340107 22707 leader_election.cc:302] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85d01da39508411cb57cc674a99db034; no voters:
I20260620 14:56:00.340483 22707 leader_election.cc:288] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:56:00.340559 22712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:56:00.340847 22712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 1 LEADER]: Becoming Leader. State: Replica: 85d01da39508411cb57cc674a99db034, State: Running, Role: LEADER
I20260620 14:56:00.341256 22712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:00.341778 22707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:56:00.343103 22714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85d01da39508411cb57cc674a99db034. Latest consensus state: current_term: 1 leader_uuid: "85d01da39508411cb57cc674a99db034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } } }
I20260620 14:56:00.343221 22714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: This master's current role is: LEADER
I20260620 14:56:00.343163 22713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85d01da39508411cb57cc674a99db034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } } }
I20260620 14:56:00.343264 22713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: This master's current role is: LEADER
I20260620 14:56:00.343704 22722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:56:00.346570 22722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:56:00.352495 22722 catalog_manager.cc:1347] Generated new cluster ID: 203cb7da662443a8a004ecf74ffc4fa7
I20260620 14:56:00.352578 22722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:56:00.361734 22722 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:56:00.362829 22722 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:56:00.368175 22722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Generated new TSK 0
I20260620 14:56:00.368896 22722 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:56:00.397156 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40807
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:56:00.544020 22733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:00.544294 22733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:00.544342 22733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:00.548846 22733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:00.548918 22733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:00.548981 22733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:56:00.554970 22733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:00.556185 22733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:00.557215 22733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:00.567441 22739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:00.567479 22741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:00.567910 22738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:00.568024 22733 server_base.cc:1034] running on GCE node
I20260620 14:56:00.568439 22733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:00.569051 22733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:00.570271 22733 hybrid_clock.cc:648] HybridClock initialized: now 1781967360570232 us; error 48 us; skew 500 ppm
Jun 20 14:56:00 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967360, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:00.573299 22733 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:56:00.574203 22733 webserver.cc:460] Webserver started at http://127.7.24.193:34029/ using document root <none> and password file <none>
I20260620 14:56:00.574862 22733 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:00.574935 22733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:00.575109 22733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:56:00.577237 22733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "d5a59927975948d196f96d42b777952e"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "c1163efc2fa0673a36d3a5401a0458af"
server_key_iv: "096f71fbb8e387577e810e7b24d08b11"
server_key_version: "encryptionkey@0"
I20260620 14:56:00.577771 22733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "d5a59927975948d196f96d42b777952e"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "c1163efc2fa0673a36d3a5401a0458af"
server_key_iv: "096f71fbb8e387577e810e7b24d08b11"
server_key_version: "encryptionkey@0"
I20260620 14:56:00.581811 22733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:56:00.584460 22748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:00.585913 22733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 14:56:00.586072 22733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "d5a59927975948d196f96d42b777952e"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "c1163efc2fa0673a36d3a5401a0458af"
server_key_iv: "096f71fbb8e387577e810e7b24d08b11"
server_key_version: "encryptionkey@0"
I20260620 14:56:00.586207 22733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:00.602002 22733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:00.605410 22733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:00.605633 22733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:00.606278 22733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:56:00.607234 22733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:56:00.607286 22733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:00.607357 22733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:56:00.607399 22733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:00.617219 22733 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:35421
I20260620 14:56:00.617241 22861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:35421 every 8 connection(s)
I20260620 14:56:00.618290 22733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:56:00.625298 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 22733
I20260620 14:56:00.625438 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:56:00.625763 7267 external_mini_cluster.cc:1428] Setting key eb3c14d6058a4d101cf98f6a302e7285
Jun 20 14:56:00 dist-test-slave-sms9 krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967360, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:56:00.633528 22710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:00.620249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:54707 (local address 127.7.24.254:40807)
0620 14:56:00.620400 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:00.620406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:00.621501 (+ 1095us) server_negotiation.cc:407] Connection header received
0620 14:56:00.622516 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:00.622520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:00.622582 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:00.622672 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:00.623897 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:00.624493 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:00.626131 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:00.626506 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:00.626598 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:00.629535 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:00.629538 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:00.629540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:00.629565 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:00.631112 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:00.631822 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:00.631827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:00.631829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:00.631929 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:00.632300 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:00.632305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:00.632307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:00.632511 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:00.632644 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:00.633212 (+ 568us) server_negotiation.cc:299] Negotiation successful
0620 14:56:00.633348 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 14:56:00.634418 22864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:00.620617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:54707)
0620 14:56:00.621297 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:00.621342 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:00.622262 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:00.622842 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:00.622849 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:00.623228 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:56:00.623731 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:00.623745 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:00.624634 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:00.624638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:00.626003 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:00.626015 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:00.626660 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:00.626665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:00.626867 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:00.627614 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:56:00.627632 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:56:00.629328 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:56:00.631256 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:00.631262 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:00.631278 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:00.631709 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:00.632040 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:00.632043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:00.632046 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:00.632195 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:00.632656 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:56:00.632665 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:56:00.632927 (+ 262us) client_negotiation.cc:769] Sending connection context
0620 14:56:00.633163 (+ 236us) client_negotiation.cc:240] Negotiation successful
0620 14:56:00.633531 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:56:00.635968 22862 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40807
I20260620 14:56:00.636260 22862 heartbeater.cc:464] Registering TS with master...
I20260620 14:56:00.636919 22862 heartbeater.cc:511] Master 127.7.24.254:40807 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
I20260620 14:56:00.638907 22671 ts_manager.cc:194] Registered new tserver with Master: d5a59927975948d196f96d42b777952e (127.7.24.193:35421)
I20260620 14:56:00.640172 22671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:54707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:56:00.662262 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40807
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:56:00.778371 22867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:00.778712 22867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:00.778792 22867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:00.783157 22867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:00.783262 22867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:00.783360 22867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:56:00.788997 22867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:00.790271 22867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:00.791465 22867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:00.801393 22875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:00.801384 22872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:00.801384 22873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:00.801815 22867 server_base.cc:1034] running on GCE node
I20260620 14:56:00.802256 22867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:00.802996 22867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:00.804176 22867 hybrid_clock.cc:648] HybridClock initialized: now 1781967360804118 us; error 74 us; skew 500 ppm
Jun 20 14:56:00 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967360, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:00.807301 22867 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:56:00.808354 22867 webserver.cc:460] Webserver started at http://127.7.24.194:44529/ using document root <none> and password file <none>
I20260620 14:56:00.809001 22867 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:00.809067 22867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:00.809240 22867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:56:00.811388 22867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "6fefd3b8f93e435c92da1cf57917994c"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "3c6500fc4166bd0a87d7a329e02e1de7"
server_key_iv: "766dcb6e2924dee3292891eef194565f"
server_key_version: "encryptionkey@0"
I20260620 14:56:00.811931 22867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "6fefd3b8f93e435c92da1cf57917994c"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "3c6500fc4166bd0a87d7a329e02e1de7"
server_key_iv: "766dcb6e2924dee3292891eef194565f"
server_key_version: "encryptionkey@0"
I20260620 14:56:00.816007 22867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260620 14:56:00.818749 22882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:00.820019 22867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 14:56:00.820127 22867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "6fefd3b8f93e435c92da1cf57917994c"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "3c6500fc4166bd0a87d7a329e02e1de7"
server_key_iv: "766dcb6e2924dee3292891eef194565f"
server_key_version: "encryptionkey@0"
I20260620 14:56:00.820276 22867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:00.838693 22867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:00.842394 22867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:00.842772 22867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:00.843572 22867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:56:00.844748 22867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:56:00.844803 22867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:00.844882 22867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:56:00.844916 22867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:00.855026 22867 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:36979
I20260620 14:56:00.855044 22995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:36979 every 8 connection(s)
I20260620 14:56:00.856093 22867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:56:00.859573 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 22867
I20260620 14:56:00.859702 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:56:00.860019 7267 external_mini_cluster.cc:1428] Setting key 164f2ad66b4c9720adfd8903ca0437cd
Jun 20 14:56:00 dist-test-slave-sms9 krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967360, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:56:00.872391 22710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:00.858099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:58031 (local address 127.7.24.254:40807)
0620 14:56:00.858300 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:00.858310 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:00.859690 (+ 1380us) server_negotiation.cc:407] Connection header received
0620 14:56:00.860876 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:00.860880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:00.860941 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:00.861042 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:00.862807 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:00.863445 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:00.864791 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:00.865046 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:00.865178 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:00.868499 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:00.868503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:00.868505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:00.868533 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:00.870052 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:00.870723 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:00.870726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:00.870727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:00.870818 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:00.871193 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:00.871197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:00.871198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:00.871410 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:00.871525 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:00.872083 (+ 558us) server_negotiation.cc:299] Negotiation successful
0620 14:56:00.872211 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
I20260620 14:56:00.873358 22998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:00.858649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:58031)
0620 14:56:00.859482 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:00.859552 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:00.860618 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:00.861238 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:00.861244 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:00.861826 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:56:00.862622 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:00.862648 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:00.863619 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:00.863623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:00.864637 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:00.864648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:00.865229 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:00.865234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:00.865430 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:00.866147 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:56:00.866166 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:56:00.868239 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:56:00.870208 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:00.870215 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:00.870235 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:00.870601 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:00.870922 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:00.870925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:00.870927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:00.871086 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:00.871524 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:56:00.871533 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:56:00.871818 (+ 285us) client_negotiation.cc:769] Sending connection context
0620 14:56:00.872025 (+ 207us) client_negotiation.cc:240] Negotiation successful
0620 14:56:00.872388 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":126,"threads_started":1}
I20260620 14:56:00.874950 22996 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40807
I20260620 14:56:00.875221 22996 heartbeater.cc:464] Registering TS with master...
I20260620 14:56:00.875770 22996 heartbeater.cc:511] Master 127.7.24.254:40807 requested a full tablet report, sending...
I20260620 14:56:00.877000 22671 ts_manager.cc:194] Registered new tserver with Master: 6fefd3b8f93e435c92da1cf57917994c (127.7.24.194:36979)
I20260620 14:56:00.877665 22671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:58031
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:56:00.890329 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40807
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:56:01.005038 23001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:01.005389 23001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:01.005529 23001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:01.010058 23001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:01.010135 23001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:01.010201 23001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:56:01.015987 23001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:01.017189 23001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:01.018226 23001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:01.028187 23006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:01.028214 23009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:01.028214 23007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:01.028421 23001 server_base.cc:1034] running on GCE node
I20260620 14:56:01.028908 23001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:01.029613 23001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:01.030805 23001 hybrid_clock.cc:648] HybridClock initialized: now 1781967361030785 us; error 44 us; skew 500 ppm
Jun 20 14:56:01 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967361, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:01.033776 23001 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:56:01.034736 23001 webserver.cc:460] Webserver started at http://127.7.24.195:40679/ using document root <none> and password file <none>
I20260620 14:56:01.035326 23001 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:01.035418 23001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:01.035657 23001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:56:01.037850 23001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "6a114f16bd2d48fc9134a99ce48f8acc"
format_stamp: "Formatted at 2026-06-20 14:56:01 on dist-test-slave-sms9"
server_key: "c2bd359e2787fa3e5c7156989f181c0a"
server_key_iv: "d51ec9b404e772ca7d3be475bbef2a0e"
server_key_version: "encryptionkey@0"
I20260620 14:56:01.038419 23001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "6a114f16bd2d48fc9134a99ce48f8acc"
format_stamp: "Formatted at 2026-06-20 14:56:01 on dist-test-slave-sms9"
server_key: "c2bd359e2787fa3e5c7156989f181c0a"
server_key_iv: "d51ec9b404e772ca7d3be475bbef2a0e"
server_key_version: "encryptionkey@0"
I20260620 14:56:01.042240 23001 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:56:01.044996 23016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:01.046196 23001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:56:01.046339 23001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "6a114f16bd2d48fc9134a99ce48f8acc"
format_stamp: "Formatted at 2026-06-20 14:56:01 on dist-test-slave-sms9"
server_key: "c2bd359e2787fa3e5c7156989f181c0a"
server_key_iv: "d51ec9b404e772ca7d3be475bbef2a0e"
server_key_version: "encryptionkey@0"
I20260620 14:56:01.046517 23001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:01.061801 23001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:01.065300 23001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:01.065526 23001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:01.066144 23001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:56:01.067058 23001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:56:01.067127 23001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:01.067198 23001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:56:01.067240 23001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:01.076051 23001 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:39543
I20260620 14:56:01.076076 23129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:39543 every 8 connection(s)
I20260620 14:56:01.076974 23001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:56:01.077678 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 23001
I20260620 14:56:01.077837 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:56:01.078133 7267 external_mini_cluster.cc:1428] Setting key e8971fb40dadd014765b7cb2b5323620
Jun 20 14:56:01 dist-test-slave-sms9 krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967361, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:56:01.091225 22710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.079012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:51993 (local address 127.7.24.254:40807)
0620 14:56:01.079146 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:01.079150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:01.080460 (+ 1310us) server_negotiation.cc:407] Connection header received
0620 14:56:01.081426 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:01.081429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:01.081475 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:01.081555 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:01.082929 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.083468 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.084723 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.084961 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.085052 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.087597 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:01.087601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:01.087603 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:01.087628 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:01.089016 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:01.089534 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:01.089536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:01.089538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:01.089611 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:01.089974 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:01.089976 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:01.089977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:01.090153 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:01.090229 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:01.090879 (+ 650us) server_negotiation.cc:299] Negotiation successful
0620 14:56:01.091049 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:56:01.092226 23132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.079528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:51993)
0620 14:56:01.080275 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:01.080350 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:01.081245 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:01.081753 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:01.081760 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:01.082151 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:56:01.082785 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.082801 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.083587 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.083590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.084595 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.084605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.085102 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.085105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.085249 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.085833 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:56:01.085856 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:56:01.087432 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:56:01.089126 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:01.089131 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:01.089144 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:01.089446 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:01.089717 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:01.089719 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:01.089721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:01.089876 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:01.090240 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:56:01.090245 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:56:01.090596 (+ 351us) client_negotiation.cc:769] Sending connection context
0620 14:56:01.090845 (+ 249us) client_negotiation.cc:240] Negotiation successful
0620 14:56:01.091195 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":157,"threads_started":1}
I20260620 14:56:01.093279 23130 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40807
I20260620 14:56:01.093540 23130 heartbeater.cc:464] Registering TS with master...
I20260620 14:56:01.094019 23130 heartbeater.cc:511] Master 127.7.24.254:40807 requested a full tablet report, sending...
I20260620 14:56:01.095021 22671 ts_manager.cc:194] Registered new tserver with Master: 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195:39543)
I20260620 14:56:01.095594 22671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:51993
I20260620 14:56:01.104357 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:56:01.114145 22710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.106319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56608 (local address 127.7.24.254:40807)
0620 14:56:01.106444 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:01.106448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:01.106629 (+ 181us) server_negotiation.cc:407] Connection header received
0620 14:56:01.106729 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:01.106731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:01.106775 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:01.106845 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:01.107795 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.108383 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.109260 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.109530 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.109617 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.111057 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:01.111060 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:01.111062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:01.111087 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:01.112503 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:01.112956 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:01.112959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:01.112960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:01.113052 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:01.113374 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:01.113377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:01.113379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:01.113606 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:01.113718 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:01.113924 (+ 206us) server_negotiation.cc:299] Negotiation successful
0620 14:56:01.114023 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:56:01.117229 22671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:56:01.120445 22671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:56:01.135017 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.129132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:35421 (local address 127.0.0.1:59090)
0620 14:56:01.129500 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:01.129545 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:01.129648 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:01.130869 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:01.130871 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:01.130893 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:56:01.130972 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.130976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.132046 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.132052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.133513 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.133524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.134396 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.134399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.134531 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.134572 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 14:56:01.134706 (+ 134us) client_negotiation.cc:240] Negotiation successful
0620 14:56:01.134823 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":60,"thread_start_us":91,"threads_started":1}
I20260620 14:56:01.135255 23143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.129125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38626 (local address 127.7.24.194:36979)
0620 14:56:01.129968 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:01.129981 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:01.130004 (+ 23us) server_negotiation.cc:407] Connection header received
0620 14:56:01.130083 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:01.130088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:01.130289 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:01.130837 (+ 548us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:56:01.131011 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.132230 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.133699 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.134544 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.134685 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.134793 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:01.134991 (+ 198us) server_negotiation.cc:299] Negotiation successful
0620 14:56:01.135106 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":108,"threads_started":1}
I20260620 14:56:01.135535 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.129993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.7.24.195:39543)
0620 14:56:01.130389 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:01.130401 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:01.130645 (+ 244us) server_negotiation.cc:407] Connection header received
0620 14:56:01.130717 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:01.130722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:01.130913 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:01.131060 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:56:01.131284 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.132390 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.133764 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.134544 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.134656 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.134783 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:01.135311 (+ 528us) server_negotiation.cc:299] Negotiation successful
0620 14:56:01.135417 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":132,"threads_started":1}
I20260620 14:56:01.135746 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.129716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59090 (local address 127.7.24.193:35421)
0620 14:56:01.130010 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:01.130026 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:01.130049 (+ 23us) server_negotiation.cc:407] Connection header received
0620 14:56:01.130130 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:01.130137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:01.130574 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:01.130725 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:56:01.131118 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.131920 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.133636 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.134257 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.134401 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.134581 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:01.135512 (+ 931us) server_negotiation.cc:299] Negotiation successful
0620 14:56:01.135591 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":104,"threads_started":1}
I20260620 14:56:01.135226 23141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.128949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:36979 (local address 127.0.0.1:38626)
0620 14:56:01.129476 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:01.129528 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:01.129647 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:01.130506 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:01.130514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:01.130544 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:56:01.130702 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.130709 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.132408 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.132412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.133608 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.133616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.134709 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.134713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.134793 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.134813 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 14:56:01.134904 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 14:56:01.135007 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":71,"spinlock_wait_cycles":2432,"thread_start_us":96,"threads_started":1}
I20260620 14:56:01.135789 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.129632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:39543 (local address 127.0.0.1:34174)
0620 14:56:01.130433 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:01.130447 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:01.130553 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:01.131056 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:01.131062 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:01.131081 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:56:01.131169 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.131175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.132531 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.132536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.133657 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.133667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.134843 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.134847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.135057 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.135082 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 14:56:01.135153 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 14:56:01.135209 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":10,"spinlock_wait_cycles":768,"thread_start_us":86,"threads_started":1}
I20260620 14:56:01.138897 22930 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4ebd42378a4fd98d00a12dea43f963 (DEFAULT_TABLE table=test-table [id=bed481c582cd4f9191a46e334aea9cab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:56:01.138995 23064 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4ebd42378a4fd98d00a12dea43f963 (DEFAULT_TABLE table=test-table [id=bed481c582cd4f9191a46e334aea9cab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:56:01.139981 22930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4ebd42378a4fd98d00a12dea43f963. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:56:01.139991 23064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4ebd42378a4fd98d00a12dea43f963. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:56:01.140359 22796 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4ebd42378a4fd98d00a12dea43f963 (DEFAULT_TABLE table=test-table [id=bed481c582cd4f9191a46e334aea9cab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:56:01.141294 22796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4ebd42378a4fd98d00a12dea43f963. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:56:01.146584 23147 tablet_bootstrap.cc:492] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: Bootstrap starting.
I20260620 14:56:01.147866 23149 tablet_bootstrap.cc:492] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Bootstrap starting.
I20260620 14:56:01.148295 23148 tablet_bootstrap.cc:492] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: Bootstrap starting.
I20260620 14:56:01.149354 23147 tablet_bootstrap.cc:654] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: Neither blocks nor log segments found. Creating new log.
I20260620 14:56:01.149713 23149 tablet_bootstrap.cc:654] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Neither blocks nor log segments found. Creating new log.
I20260620 14:56:01.150463 23149 log.cc:824] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Log is configured to *not* fsync() on all Append() calls
I20260620 14:56:01.150494 23147 log.cc:824] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: Log is configured to *not* fsync() on all Append() calls
I20260620 14:56:01.150889 23148 tablet_bootstrap.cc:654] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: Neither blocks nor log segments found. Creating new log.
I20260620 14:56:01.151705 23148 log.cc:824] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: Log is configured to *not* fsync() on all Append() calls
I20260620 14:56:01.152659 23149 tablet_bootstrap.cc:492] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: No bootstrap required, opened a new log
I20260620 14:56:01.152840 23147 tablet_bootstrap.cc:492] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: No bootstrap required, opened a new log
I20260620 14:56:01.152916 23149 ts_tablet_manager.cc:1397] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 14:56:01.153023 23147 ts_tablet_manager.cc:1397] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260620 14:56:01.153558 23148 tablet_bootstrap.cc:492] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: No bootstrap required, opened a new log
I20260620 14:56:01.153734 23148 ts_tablet_manager.cc:1397] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260620 14:56:01.156539 23149 raft_consensus.cc:348] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } }
I20260620 14:56:01.156801 23149 raft_consensus.cc:374] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:56:01.156877 23149 raft_consensus.cc:729] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a59927975948d196f96d42b777952e, State: Initialized, Role: FOLLOWER
I20260620 14:56:01.156914 23147 raft_consensus.cc:348] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } }
I20260620 14:56:01.157411 23147 raft_consensus.cc:374] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:56:01.157517 23147 raft_consensus.cc:729] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a114f16bd2d48fc9134a99ce48f8acc, State: Initialized, Role: FOLLOWER
I20260620 14:56:01.157582 23149 consensus_queue.cc:260] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } }
I20260620 14:56:01.157526 23148 raft_consensus.cc:348] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } }
I20260620 14:56:01.157763 23148 raft_consensus.cc:374] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:56:01.157843 23148 raft_consensus.cc:729] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fefd3b8f93e435c92da1cf57917994c, State: Initialized, Role: FOLLOWER
I20260620 14:56:01.158108 23147 consensus_queue.cc:260] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } }
I20260620 14:56:01.158273 23148 consensus_queue.cc:260] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } }
I20260620 14:56:01.158825 23149 ts_tablet_manager.cc:1428] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260620 14:56:01.159039 22996 heartbeater.cc:503] Master 127.7.24.254:40807 was elected leader, sending a full tablet report...
I20260620 14:56:01.159092 23147 ts_tablet_manager.cc:1428] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 14:56:01.159198 22862 heartbeater.cc:503] Master 127.7.24.254:40807 was elected leader, sending a full tablet report...
I20260620 14:56:01.159457 23130 heartbeater.cc:503] Master 127.7.24.254:40807 was elected leader, sending a full tablet report...
I20260620 14:56:01.160167 23148 ts_tablet_manager.cc:1428] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260620 14:56:01.327950 23131 tablet.cc:2369] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:56:01.334236 23153 raft_consensus.cc:482] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:56:01.334445 23153 raft_consensus.cc:504] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } }
I20260620 14:56:01.335639 23153 leader_election.cc:288] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fefd3b8f93e435c92da1cf57917994c (127.7.24.194:36979), 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195:39543)
I20260620 14:56:01.340360 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.336016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:36979 (local address 127.7.24.193:38939)
0620 14:56:01.336329 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:01.336347 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:01.336469 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:01.336959 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:01.336963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:01.336985 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:56:01.337128 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.337136 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.338039 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.338043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.339049 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.339059 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.339855 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.339859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.339964 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.339992 (+ 28us) client_negotiation.cc:769] Sending connection context
0620 14:56:01.340097 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 14:56:01.340184 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":127,"threads_started":1}
I20260620 14:56:01.340349 23143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.336106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:38939 (local address 127.7.24.194:36979)
0620 14:56:01.336252 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:01.336256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:01.336455 (+ 199us) server_negotiation.cc:407] Connection header received
0620 14:56:01.336752 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:01.336755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:01.336823 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:01.336924 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:56:01.337245 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.337826 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.339188 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.339720 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.339820 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.339901 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:01.340119 (+ 218us) server_negotiation.cc:299] Negotiation successful
0620 14:56:01.340209 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 14:56:01.340865 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.336625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:50213 (local address 127.7.24.195:39543)
0620 14:56:01.336777 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:01.336780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:01.336838 (+ 58us) server_negotiation.cc:407] Connection header received
0620 14:56:01.337004 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:01.337007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:01.337057 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:01.337138 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:56:01.337451 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.338107 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.339546 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.340126 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.340237 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.340283 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:01.340621 (+ 338us) server_negotiation.cc:299] Negotiation successful
0620 14:56:01.340741 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 14:56:01.341238 23157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.336459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:39543 (local address 127.7.24.193:50213)
0620 14:56:01.336760 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:01.336773 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:01.336885 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:01.337159 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:01.337161 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:01.337176 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:56:01.337321 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.337328 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.338248 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.338252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.339277 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:01.339288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.340244 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.340248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:01.340438 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.340471 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 14:56:01.341022 (+ 551us) client_negotiation.cc:240] Negotiation successful
0620 14:56:01.341101 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":95,"threads_started":1}
I20260620 14:56:01.341260 22950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4ebd42378a4fd98d00a12dea43f963" candidate_uuid: "d5a59927975948d196f96d42b777952e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fefd3b8f93e435c92da1cf57917994c" is_pre_election: true
I20260620 14:56:01.341620 22950 raft_consensus.cc:2454] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a59927975948d196f96d42b777952e in term 0.
I20260620 14:56:01.341920 23084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4ebd42378a4fd98d00a12dea43f963" candidate_uuid: "d5a59927975948d196f96d42b777952e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" is_pre_election: true
I20260620 14:56:01.342242 22749 leader_election.cc:302] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fefd3b8f93e435c92da1cf57917994c, d5a59927975948d196f96d42b777952e; no voters:
I20260620 14:56:01.342262 23084 raft_consensus.cc:2454] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a59927975948d196f96d42b777952e in term 0.
I20260620 14:56:01.342644 23153 raft_consensus.cc:2781] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:56:01.342761 23153 raft_consensus.cc:482] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:56:01.342834 23153 raft_consensus.cc:3037] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:56:01.344128 23153 raft_consensus.cc:504] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } }
I20260620 14:56:01.344553 23153 leader_election.cc:288] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [CANDIDATE]: Term 1 election: Requested vote from peers 6fefd3b8f93e435c92da1cf57917994c (127.7.24.194:36979), 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195:39543)
I20260620 14:56:01.345008 23084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4ebd42378a4fd98d00a12dea43f963" candidate_uuid: "d5a59927975948d196f96d42b777952e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a114f16bd2d48fc9134a99ce48f8acc"
I20260620 14:56:01.345024 22950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4ebd42378a4fd98d00a12dea43f963" candidate_uuid: "d5a59927975948d196f96d42b777952e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fefd3b8f93e435c92da1cf57917994c"
I20260620 14:56:01.345124 23084 raft_consensus.cc:3037] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:56:01.345165 22950 raft_consensus.cc:3037] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:56:01.346271 23084 raft_consensus.cc:2454] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a59927975948d196f96d42b777952e in term 1.
I20260620 14:56:01.346271 22950 raft_consensus.cc:2454] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a59927975948d196f96d42b777952e in term 1.
I20260620 14:56:01.346650 22749 leader_election.cc:302] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a114f16bd2d48fc9134a99ce48f8acc, d5a59927975948d196f96d42b777952e; no voters:
I20260620 14:56:01.346879 23153 raft_consensus.cc:2781] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:56:01.347136 23153 raft_consensus.cc:686] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 1 LEADER]: Becoming Leader. State: Replica: d5a59927975948d196f96d42b777952e, State: Running, Role: LEADER
I20260620 14:56:01.347570 23153 consensus_queue.cc:237] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } }
I20260620 14:56:01.351256 22670 catalog_manager.cc:5495] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e reported cstate change: term changed from 0 to 1, leader changed from <none> to d5a59927975948d196f96d42b777952e (127.7.24.193). New cstate: current_term: 1 leader_uuid: "d5a59927975948d196f96d42b777952e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } health_report { overall_health: UNKNOWN } } }
W20260620 14:56:01.357321 22997 tablet.cc:2369] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 14:56:01.369799 22863 tablet.cc:2369] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:56:01.371595 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.367867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59106 (local address 127.7.24.193:35421)
0620 14:56:01.368019 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:01.368023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:01.368040 (+ 17us) server_negotiation.cc:407] Connection header received
0620 14:56:01.368135 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:01.368137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:01.368195 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:01.368310 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:56:01.368599 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.369111 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.370096 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:01.370311 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:01.370444 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:01.370738 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:56:01.371149 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:56:01.371243 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:01.371309 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 14:56:01.371362 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 14:56:01.377465 22950 raft_consensus.cc:1261] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Refusing update from remote peer d5a59927975948d196f96d42b777952e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:56:01.377471 23084 raft_consensus.cc:1261] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc [term 1 FOLLOWER]: Refusing update from remote peer d5a59927975948d196f96d42b777952e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:56:01.378235 23153 consensus_queue.cc:1035] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:56:01.378474 23158 consensus_queue.cc:1035] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:56:01.387722 23162 mvcc.cc:205] Tried to move back new op lower bound from 7298938312194772992 to 7298938312082632704. Current Snapshot: MvccSnapshot[applied={T|T < 7298938312194772992}]
I20260620 14:56:01.394717 22671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:56:01.394883 22671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:56:01.397444 22671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Sending DeleteTablet for 3 replicas of tablet 7a4ebd42378a4fd98d00a12dea43f963
I20260620 14:56:01.398319 22796 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a4ebd42378a4fd98d00a12dea43f963 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:56:01 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:59090
I20260620 14:56:01.398347 23064 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a4ebd42378a4fd98d00a12dea43f963 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:56:01 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:34174
I20260620 14:56:01.398979 23171 tablet_replica.cc:330] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: stopping tablet replica
I20260620 14:56:01.399168 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 22733
I20260620 14:56:01.399137 22930 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a4ebd42378a4fd98d00a12dea43f963 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:56:01 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:38626
I20260620 14:56:01.399569 23173 tablet_replica.cc:330] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: stopping tablet replica
I20260620 14:56:01.399956 23173 raft_consensus.cc:2229] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:56:01.400293 23173 raft_consensus.cc:2258] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:56:01.400470 23171 raft_consensus.cc:2229] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:56:01.400739 23171 raft_consensus.cc:2258] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:56:01.401959 23173 ts_tablet_manager.cc:1905] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:56:01.401973 23171 ts_tablet_manager.cc:1905] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:56:01.404711 23171 ts_tablet_manager.cc:1918] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:56:01.404711 23173 ts_tablet_manager.cc:1918] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:56:01.404815 23171 log.cc:1196] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/7a4ebd42378a4fd98d00a12dea43f963
I20260620 14:56:01.404815 23173 log.cc:1196] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/7a4ebd42378a4fd98d00a12dea43f963
I20260620 14:56:01.405151 23173 ts_tablet_manager.cc:1939] T 7a4ebd42378a4fd98d00a12dea43f963 P 6fefd3b8f93e435c92da1cf57917994c: Deleting consensus metadata
I20260620 14:56:01.405164 23171 ts_tablet_manager.cc:1939] T 7a4ebd42378a4fd98d00a12dea43f963 P 6a114f16bd2d48fc9134a99ce48f8acc: Deleting consensus metadata
I20260620 14:56:01.406607 22655 catalog_manager.cc:4841] TS 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195:39543): tablet 7a4ebd42378a4fd98d00a12dea43f963 (table test-table [id=bed481c582cd4f9191a46e334aea9cab]) successfully deleted
I20260620 14:56:01.406989 22655 catalog_manager.cc:4841] TS 6fefd3b8f93e435c92da1cf57917994c (127.7.24.194:36979): tablet 7a4ebd42378a4fd98d00a12dea43f963 (table test-table [id=bed481c582cd4f9191a46e334aea9cab]) successfully deleted
I20260620 14:56:01.407264 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 22867
W20260620 14:56:01.407254 22657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:35421 (error 108)
I20260620 14:56:01.408677 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:01.408345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:35421 (local address 127.0.0.1:59112)
0620 14:56:01.408532 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:01.408601 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:35421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260620 14:56:01.408813 22657 catalog_manager.cc:4584] TS d5a59927975948d196f96d42b777952e (127.7.24.193:35421): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a4ebd42378a4fd98d00a12dea43f963: Network error: Client connection negotiation failed: client connection to 127.7.24.193:35421: connect: Connection refused (error 111)
I20260620 14:56:01.414302 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 23001
I20260620 14:56:01.420126 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 22639
I20260620 14:56:21.427147 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40807
--webserver_interface=127.7.24.254
--webserver_port=38021
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:56:21.541680 23175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:21.541980 23175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:21.542023 23175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:21.546227 23175 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 14:56:21.546296 23175 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 14:56:21.546314 23175 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 14:56:21.546329 23175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:56:21.546425 23175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:21.546479 23175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:21.546510 23175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:56:21.546754 23175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:56:21.552376 23175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=38021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:21.553489 23175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:21.554543 23175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:21.568225 23182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:21.568336 23175 server_base.cc:1034] running on GCE node
W20260620 14:56:21.568640 23184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:21.568225 23181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:21.569624 23175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:21.571317 23175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:21.574554 23175 hybrid_clock.cc:648] HybridClock initialized: now 1781967381574351 us; error 219 us; skew 500 ppm
Jun 20 14:56:21 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:21.578719 23175 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:56:21.580003 23175 webserver.cc:460] Webserver started at http://127.7.24.254:38021/ using document root <none> and password file <none>
I20260620 14:56:21.580878 23175 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:21.580991 23175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:21.586112 23175 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:56:21.588811 23191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:21.590720 23175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260620 14:56:21.590860 23175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "85d01da39508411cb57cc674a99db034"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "ea45a129d3fc3304a1d3d847f3284267"
server_key_iv: "104e759ff72f1db40225e2bd0ede3bc7"
server_key_version: "encryptionkey@0"
I20260620 14:56:21.591679 23175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:21.635798 23175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:21.639252 23175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:21.639472 23175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:21.646641 23175 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:40807
I20260620 14:56:21.646951 23243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:40807 every 8 connection(s)
I20260620 14:56:21.647750 23175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:56:21.652123 23244 sys_catalog.cc:263] Verifying existing consensus state
I20260620 14:56:21.653743 23244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Bootstrap starting.
I20260620 14:56:21.654695 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 23175
I20260620 14:56:21.655153 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:35421
--local_ip_for_outbound_sockets=127.7.24.193
--tserver_master_addrs=127.7.24.254:40807
--webserver_port=34029
--webserver_interface=127.7.24.193
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:56:21.671084 23244 log.cc:824] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Log is configured to *not* fsync() on all Append() calls
I20260620 14:56:21.679822 23244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:56:21.680210 23244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Bootstrap complete.
I20260620 14:56:21.684439 23244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:21.684828 23244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d01da39508411cb57cc674a99db034, State: Initialized, Role: FOLLOWER
I20260620 14:56:21.685308 23244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:21.685451 23244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:56:21.685523 23244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:56:21.685649 23244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 1 FOLLOWER]: Advancing to term 2
I20260620 14:56:21.687602 23244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:21.687935 23244 leader_election.cc:302] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85d01da39508411cb57cc674a99db034; no voters:
I20260620 14:56:21.688261 23244 leader_election.cc:288] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 14:56:21.688350 23248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 14:56:21.688548 23248 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 2 LEADER]: Becoming Leader. State: Replica: 85d01da39508411cb57cc674a99db034, State: Running, Role: LEADER
I20260620 14:56:21.688835 23248 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:21.689538 23244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:56:21.690200 23250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "85d01da39508411cb57cc674a99db034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } } }
I20260620 14:56:21.690634 23250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: This master's current role is: LEADER
I20260620 14:56:21.690537 23249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85d01da39508411cb57cc674a99db034. Latest consensus state: current_term: 2 leader_uuid: "85d01da39508411cb57cc674a99db034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } } }
I20260620 14:56:21.690961 23249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: This master's current role is: LEADER
I20260620 14:56:21.691306 23258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:56:21.696079 23258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:56:21.696935 23258 catalog_manager.cc:1259] Loaded cluster ID: 203cb7da662443a8a004ecf74ffc4fa7
I20260620 14:56:21.696993 23258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:56:21.699592 23258 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:56:21.700434 23258 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Loaded TSK: 0
I20260620 14:56:21.719986 23258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Generated new TSK 1
I20260620 14:56:21.720278 23258 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 14:56:21.774076 23246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:21.774315 23246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:21.774358 23246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:21.778847 23246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:21.778927 23246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:21.779028 23246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:56:21.784317 23246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:35421
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=34029
--tserver_master_addrs=127.7.24.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:21.785290 23246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:21.786252 23246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:21.795982 23274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:21.795982 23271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:21.795982 23272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:21.796416 23246 server_base.cc:1034] running on GCE node
I20260620 14:56:21.796805 23246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:21.797513 23246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:21.798684 23246 hybrid_clock.cc:648] HybridClock initialized: now 1781967381798669 us; error 29 us; skew 500 ppm
Jun 20 14:56:21 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:21.801558 23246 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:56:21.802398 23246 webserver.cc:460] Webserver started at http://127.7.24.193:34029/ using document root <none> and password file <none>
I20260620 14:56:21.803009 23246 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:21.803122 23246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:21.807060 23246 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:56:21.809196 23281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:21.810595 23246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:56:21.810720 23246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "d5a59927975948d196f96d42b777952e"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "c1163efc2fa0673a36d3a5401a0458af"
server_key_iv: "096f71fbb8e387577e810e7b24d08b11"
server_key_version: "encryptionkey@0"
I20260620 14:56:21.811470 23246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:21.823047 23246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:21.826437 23246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:21.826710 23246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:21.827307 23246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:56:21.828423 23288 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 14:56:21.833666 23246 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 14:56:21.833757 23246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260620 14:56:21.833848 23246 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 14:56:21.835613 23246 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 14:56:21.835691 23246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260620 14:56:21.835955 23288 tablet_bootstrap.cc:492] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Bootstrap starting.
I20260620 14:56:21.846175 23246 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:35421
I20260620 14:56:21.846252 23395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:35421 every 8 connection(s)
I20260620 14:56:21.847259 23246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:56:21.851744 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 23246
I20260620 14:56:21.852219 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:36979
--local_ip_for_outbound_sockets=127.7.24.194
--tserver_master_addrs=127.7.24.254:40807
--webserver_port=44529
--webserver_interface=127.7.24.194
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:56:21.856762 23288 log.cc:824] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Log is configured to *not* fsync() on all Append() calls
Jun 20 14:56:21 dist-test-slave-sms9 krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:56:21.866586 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:21.849653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:51725 (local address 127.7.24.254:40807)
0620 14:56:21.850121 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:21.850144 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:21.850476 (+ 332us) server_negotiation.cc:407] Connection header received
0620 14:56:21.851700 (+ 1224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:21.851714 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:21.852108 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:21.852500 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:21.854607 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:21.855801 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:21.857219 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:21.857968 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:21.858173 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:21.860921 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:21.860927 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:21.860940 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:21.860971 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:21.863075 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:21.863696 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:21.863701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:21.863705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:21.863837 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:21.864252 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:21.864256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:21.864258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:21.864628 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:21.864798 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:21.865351 (+ 553us) server_negotiation.cc:299] Negotiation successful
0620 14:56:21.865629 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":145,"threads_started":1}
I20260620 14:56:21.866694 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:21.849610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:51725)
0620 14:56:21.850252 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:21.850307 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:21.851220 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:21.853090 (+ 1870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:21.853100 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:21.853599 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:56:21.854353 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:21.854375 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:21.855964 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:21.855970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:21.857078 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:21.857089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:21.858133 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:21.858137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:21.858338 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:21.859084 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:56:21.859102 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:56:21.860667 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:56:21.863236 (+ 2569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:21.863242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:21.863256 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:21.863550 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:21.863945 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:21.863950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:21.863952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:21.864109 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:21.864759 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:56:21.864765 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:56:21.865027 (+ 262us) client_negotiation.cc:769] Sending connection context
0620 14:56:21.865213 (+ 186us) client_negotiation.cc:240] Negotiation successful
0620 14:56:21.865591 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":158,"threads_started":1}
I20260620 14:56:21.866730 23288 tablet_bootstrap.cc:492] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:56:21.867172 23288 tablet_bootstrap.cc:492] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Bootstrap complete.
I20260620 14:56:21.868599 23288 ts_tablet_manager.cc:1397] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Time spent bootstrapping tablet: real 0.033s user 0.030s sys 0.000s
I20260620 14:56:21.869504 23396 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40807
I20260620 14:56:21.869750 23396 heartbeater.cc:464] Registering TS with master...
I20260620 14:56:21.870262 23396 heartbeater.cc:511] Master 127.7.24.254:40807 requested a full tablet report, sending...
I20260620 14:56:21.871574 23288 raft_consensus.cc:348] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } }
I20260620 14:56:21.872213 23288 raft_consensus.cc:729] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a59927975948d196f96d42b777952e, State: Initialized, Role: FOLLOWER
I20260620 14:56:21.872690 23288 consensus_queue.cc:260] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } } peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } }
I20260620 14:56:21.872843 23208 ts_manager.cc:194] Registered new tserver with Master: d5a59927975948d196f96d42b777952e (127.7.24.193:35421)
I20260620 14:56:21.873684 23288 ts_tablet_manager.cc:1428] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 14:56:21.876514 23208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:51725
I20260620 14:56:21.879233 23396 heartbeater.cc:503] Master 127.7.24.254:40807 was elected leader, sending a full tablet report...
Jun 20 14:56:21 dist-test-slave-sms9 krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:56:21.887815 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:21.875948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:35421 (local address 127.0.0.1:45808)
0620 14:56:21.876552 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:21.876601 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:21.876897 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:21.877695 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:21.877701 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:21.878167 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:56:21.878305 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:21.878314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:21.879669 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:21.879676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:21.880771 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:21.880782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:21.881552 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:21.881556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:21.881681 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:21.882303 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:56:21.882316 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:56:21.883834 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:56:21.885884 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:21.885893 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:21.885899 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:21.886147 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:21.886512 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:21.886516 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:21.886518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:21.886610 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:21.887112 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:56:21.887118 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:56:21.887252 (+ 134us) client_negotiation.cc:769] Sending connection context
0620 14:56:21.887402 (+ 150us) client_negotiation.cc:240] Negotiation successful
0620 14:56:21.887606 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":116,"threads_started":1}
I20260620 14:56:21.887815 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:21.876030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45808 (local address 127.7.24.193:35421)
0620 14:56:21.876714 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:21.876724 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:21.876749 (+ 25us) server_negotiation.cc:407] Connection header received
0620 14:56:21.877078 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:21.877083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:21.877288 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:21.877425 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:21.878450 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:21.879562 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:21.880909 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:21.881414 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:21.881550 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:21.883981 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:21.883991 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:21.884005 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:21.884039 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:21.885737 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:21.886270 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:21.886276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:21.886278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:21.886368 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:21.886718 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:21.886722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:21.886723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:21.886989 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:21.887137 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:21.887485 (+ 348us) server_negotiation.cc:299] Negotiation successful
0620 14:56:21.887659 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":119,"threads_started":1}
I20260620 14:56:21.889050 23330 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a4ebd42378a4fd98d00a12dea43f963 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:56:01 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:45808
I20260620 14:56:21.889989 23405 tablet_replica.cc:330] stopping tablet replica
I20260620 14:56:21.890302 23405 raft_consensus.cc:2229] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:56:21.890599 23405 raft_consensus.cc:2258] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:56:21.891709 23405 ts_tablet_manager.cc:1905] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:56:21.895587 23405 ts_tablet_manager.cc:1918] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:56:21.895669 23405 log.cc:1196] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal/wals/7a4ebd42378a4fd98d00a12dea43f963
I20260620 14:56:21.895941 23405 ts_tablet_manager.cc:1939] T 7a4ebd42378a4fd98d00a12dea43f963 P d5a59927975948d196f96d42b777952e: Deleting consensus metadata
I20260620 14:56:21.897608 23194 catalog_manager.cc:4841] TS d5a59927975948d196f96d42b777952e (127.7.24.193:35421): tablet 7a4ebd42378a4fd98d00a12dea43f963 (table test-table [id=bed481c582cd4f9191a46e334aea9cab]) successfully deleted
W20260620 14:56:21.971194 23400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:21.971451 23400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:21.971501 23400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:21.975708 23400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:21.975775 23400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:21.975853 23400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:56:21.982142 23400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:36979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=44529
--tserver_master_addrs=127.7.24.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:21.983260 23400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:21.984248 23400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:21.993892 23410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:21.993974 23413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:21.993974 23411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:21.994343 23400 server_base.cc:1034] running on GCE node
I20260620 14:56:21.994760 23400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:21.995424 23400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:21.996616 23400 hybrid_clock.cc:648] HybridClock initialized: now 1781967381996596 us; error 38 us; skew 500 ppm
Jun 20 14:56:21 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:21.999576 23400 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:56:22.000439 23400 webserver.cc:460] Webserver started at http://127.7.24.194:44529/ using document root <none> and password file <none>
I20260620 14:56:22.001034 23400 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:22.001140 23400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:22.005208 23400 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:56:22.007373 23420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:22.008703 23400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:56:22.008841 23400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "6fefd3b8f93e435c92da1cf57917994c"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "3c6500fc4166bd0a87d7a329e02e1de7"
server_key_iv: "766dcb6e2924dee3292891eef194565f"
server_key_version: "encryptionkey@0"
I20260620 14:56:22.009383 23400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:22.026638 23400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:22.029742 23400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:22.029949 23400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:22.030594 23400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:56:22.031417 23400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:56:22.031495 23400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:22.031571 23400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:56:22.031625 23400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:22.041061 23400 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:36979
I20260620 14:56:22.041141 23533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:36979 every 8 connection(s)
I20260620 14:56:22.042073 23400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:56:22.049827 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 23400
I20260620 14:56:22.050351 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:39543
--local_ip_for_outbound_sockets=127.7.24.195
--tserver_master_addrs=127.7.24.254:40807
--webserver_port=40679
--webserver_interface=127.7.24.195
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 14:56:22 dist-test-slave-sms9 krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:56:22.057199 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.043976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:48363 (local address 127.7.24.254:40807)
0620 14:56:22.044121 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:22.044125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:22.045095 (+ 970us) server_negotiation.cc:407] Connection header received
0620 14:56:22.045997 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:22.046001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:22.046051 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:22.046131 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:22.047441 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.047970 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.049564 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.049796 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.049880 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.053205 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:22.053214 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:22.053216 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:22.053241 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:22.055147 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:22.055686 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:22.055691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:22.055693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:22.055776 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:22.056114 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:22.056119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:22.056121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:22.056302 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:22.056408 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:22.056893 (+ 485us) server_negotiation.cc:299] Negotiation successful
0620 14:56:22.057029 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 14:56:22.058117 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.044283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:48363)
0620 14:56:22.044941 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:22.044981 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:22.045779 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:22.046294 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:22.046345 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:22.046778 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:56:22.047252 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.047264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.048090 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.048093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.049396 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.049406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.049967 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.049971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.050172 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.050948 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:56:22.050976 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:56:22.053017 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:56:22.055282 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:22.055288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:22.055304 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:22.055579 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:22.055869 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:22.055872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:22.055874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:22.056018 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:22.056406 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:56:22.056413 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:56:22.056663 (+ 250us) client_negotiation.cc:769] Sending connection context
0620 14:56:22.056855 (+ 192us) client_negotiation.cc:240] Negotiation successful
0620 14:56:22.057205 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":106,"threads_started":1}
I20260620 14:56:22.059705 23534 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40807
I20260620 14:56:22.060072 23534 heartbeater.cc:464] Registering TS with master...
I20260620 14:56:22.060591 23534 heartbeater.cc:511] Master 127.7.24.254:40807 requested a full tablet report, sending...
I20260620 14:56:22.061838 23208 ts_manager.cc:194] Registered new tserver with Master: 6fefd3b8f93e435c92da1cf57917994c (127.7.24.194:36979)
I20260620 14:56:22.062446 23208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:48363
W20260620 14:56:22.164594 23537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:22.164948 23537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:22.165035 23537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:22.169184 23537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:22.169279 23537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:22.169406 23537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:56:22.175130 23537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:39543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=40679
--tserver_master_addrs=127.7.24.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:22.176554 23537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:22.177589 23537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:22.187022 23542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:22.187160 23543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:22.187116 23537 server_base.cc:1034] running on GCE node
W20260620 14:56:22.187024 23545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:22.187779 23537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:22.188385 23537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:22.189561 23537 hybrid_clock.cc:648] HybridClock initialized: now 1781967382189545 us; error 28 us; skew 500 ppm
Jun 20 14:56:22 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:22.192718 23537 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:56:22.193640 23537 webserver.cc:460] Webserver started at http://127.7.24.195:40679/ using document root <none> and password file <none>
I20260620 14:56:22.194231 23537 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:22.194350 23537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:22.199034 23537 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 14:56:22.201128 23552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:22.202567 23537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:56:22.202714 23537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "6a114f16bd2d48fc9134a99ce48f8acc"
format_stamp: "Formatted at 2026-06-20 14:56:01 on dist-test-slave-sms9"
server_key: "c2bd359e2787fa3e5c7156989f181c0a"
server_key_iv: "d51ec9b404e772ca7d3be475bbef2a0e"
server_key_version: "encryptionkey@0"
I20260620 14:56:22.203272 23537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:22.229771 23537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:22.232997 23537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:22.233218 23537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:22.233824 23537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:56:22.234719 23537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:56:22.234791 23537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:22.234860 23537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:56:22.234910 23537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:22.244166 23537 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:39543
I20260620 14:56:22.244180 23665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:39543 every 8 connection(s)
I20260620 14:56:22.245213 23537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:56:22.248178 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 23537
Jun 20 14:56:22 dist-test-slave-sms9 krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967382, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:56:22.258100 23670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.248825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56688 (local address 127.7.24.254:40807)
0620 14:56:22.249099 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:22.249110 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:22.249217 (+ 107us) server_negotiation.cc:407] Connection header received
0620 14:56:22.249372 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:22.249376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:22.249449 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:22.249542 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:22.250520 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.251164 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.252299 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.252590 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.252672 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.254611 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:22.254614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:22.254616 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:22.254638 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:22.256198 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:22.256763 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:22.256768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:22.256770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:22.256868 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:22.257179 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:22.257182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:22.257183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:22.257354 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:22.257509 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:22.257747 (+ 238us) server_negotiation.cc:299] Negotiation successful
0620 14:56:22.257871 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":213,"spinlock_wait_cycles":39936,"thread_start_us":100,"threads_started":1}
I20260620 14:56:22.259883 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.247041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:49935 (local address 127.7.24.254:40807)
0620 14:56:22.247174 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:22.247177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:22.248326 (+ 1149us) server_negotiation.cc:407] Connection header received
0620 14:56:22.249372 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:22.249376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:22.249449 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:22.249542 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:22.250984 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.251538 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.252959 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.253153 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.253228 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.255909 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:22.255913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:22.255915 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:22.255948 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:22.257558 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:22.258215 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:22.258220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:22.258223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:22.258321 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:22.258776 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:22.258779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:22.258780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:22.258981 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:22.259093 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:22.259606 (+ 513us) server_negotiation.cc:299] Negotiation successful
0620 14:56:22.259723 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":48,"spinlock_wait_cycles":8704}
I20260620 14:56:22.260663 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.247357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:49935)
0620 14:56:22.248140 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:22.248199 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:22.249171 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:22.249744 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:22.249754 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:22.250219 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:56:22.250852 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.250865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.251661 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.251665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.252838 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.252850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.253300 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.253307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.253465 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.254028 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:56:22.254044 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:56:22.255712 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:56:22.257701 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:22.257706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:22.257721 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:22.258058 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:22.258450 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:22.258478 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:22.258480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:22.258672 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:22.259113 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:56:22.259118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:56:22.259350 (+ 232us) client_negotiation.cc:769] Sending connection context
0620 14:56:22.259541 (+ 191us) client_negotiation.cc:240] Negotiation successful
0620 14:56:22.259862 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":128,"threads_started":1}
I20260620 14:56:22.261693 23666 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40807
I20260620 14:56:22.261970 23666 heartbeater.cc:464] Registering TS with master...
I20260620 14:56:22.262594 23666 heartbeater.cc:511] Master 127.7.24.254:40807 requested a full tablet report, sending...
I20260620 14:56:22.263592 23208 ts_manager.cc:194] Registered new tserver with Master: 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195:39543)
I20260620 14:56:22.264132 23208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:49935
I20260620 14:56:22.271713 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:56:22.275943 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.272312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56696 (local address 127.7.24.254:40807)
0620 14:56:22.272513 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:22.272518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:22.272686 (+ 168us) server_negotiation.cc:407] Connection header received
0620 14:56:22.272885 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:22.272887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:22.273008 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:22.273120 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:56:22.273360 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.273853 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.274777 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.274999 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.275106 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.275425 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:56:22.275551 (+ 126us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0620 14:56:22.275806 (+ 255us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56696: token expired
Metrics: {"server-negotiator.queue_time_us":114}
W20260620 14:56:22.276026 23398 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56696: token expired
W20260620 14:56:22.276270 23134 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.24.254:40807)
I20260620 14:56:22.283674 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.276622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.7.24.254:40807)
0620 14:56:22.276739 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:22.276745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:22.276759 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:56:22.276884 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:22.276887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:22.276954 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:22.277033 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:22.277669 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.278140 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.278979 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.279205 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.279311 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.280632 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:22.280635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:22.280638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:22.280662 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:22.281939 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:22.282375 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:22.282379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:22.282381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:22.282519 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:22.282814 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:22.282818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:22.282820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:22.282998 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:22.283134 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:22.283403 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 14:56:22.283516 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:56:22.286180 23208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:56:22.289565 23208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:56:22.297704 23330 tablet_service.cc:1467] Processing CreateTablet for tablet bb0cb22f98ef4d4cae5456d5880031b9 (DEFAULT_TABLE table=test-table [id=1935f6f06c68476793a1b61f97de4723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:56:22.298491 23330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb0cb22f98ef4d4cae5456d5880031b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:56:22.301606 23288 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: Bootstrap starting.
I20260620 14:56:22.302277 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.296379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:39543 (local address 127.0.0.1:45768)
0620 14:56:22.297060 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:22.297078 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:22.297163 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:22.297701 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:22.297704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:22.297720 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:56:22.297832 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.297842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.299007 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.299012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.300494 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.300505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.301777 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.301781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.301907 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.301962 (+ 55us) client_negotiation.cc:769] Sending connection context
0620 14:56:22.302026 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 14:56:22.302123 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597}
I20260620 14:56:22.302662 23288 tablet_bootstrap.cc:654] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: Neither blocks nor log segments found. Creating new log.
I20260620 14:56:22.302981 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.296595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.7.24.195:39543)
0620 14:56:22.297095 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:22.297110 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:22.297137 (+ 27us) server_negotiation.cc:407] Connection header received
0620 14:56:22.297316 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:22.297322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:22.297556 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:22.297713 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:56:22.297989 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.298889 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.300645 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.301631 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.302492 (+ 861us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.302669 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:22.302732 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 14:56:22.302863 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":163,"threads_started":1}
I20260620 14:56:22.303112 23677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.296845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41272 (local address 127.7.24.194:36979)
0620 14:56:22.297642 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:22.297654 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:22.297677 (+ 23us) server_negotiation.cc:407] Connection header received
0620 14:56:22.297759 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:22.297764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:22.297958 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:22.298115 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:56:22.298640 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.299807 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.301037 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.301812 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.301915 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.302059 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:22.302793 (+ 734us) server_negotiation.cc:299] Negotiation successful
0620 14:56:22.302905 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":90,"threads_started":1}
I20260620 14:56:22.303509 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.296696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:36979 (local address 127.0.0.1:41272)
0620 14:56:22.297241 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:22.297255 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:22.297338 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:22.298317 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:22.298322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:22.298341 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:56:22.298438 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.298445 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.299955 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.299958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.300918 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.300926 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.302126 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.302130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.302225 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.302248 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 14:56:22.303271 (+ 1023us) client_negotiation.cc:240] Negotiation successful
0620 14:56:22.303374 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":111,"threads_started":1}
I20260620 14:56:22.304579 23288 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: No bootstrap required, opened a new log
I20260620 14:56:22.304697 23288 ts_tablet_manager.cc:1397] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260620 14:56:22.305214 23288 raft_consensus.cc:348] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:22.305382 23288 raft_consensus.cc:374] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:56:22.305440 23288 raft_consensus.cc:729] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a59927975948d196f96d42b777952e, State: Initialized, Role: FOLLOWER
I20260620 14:56:22.305579 23288 consensus_queue.cc:260] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:22.306144 23288 ts_tablet_manager.cc:1428] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 14:56:22.306255 23600 tablet_service.cc:1467] Processing CreateTablet for tablet bb0cb22f98ef4d4cae5456d5880031b9 (DEFAULT_TABLE table=test-table [id=1935f6f06c68476793a1b61f97de4723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:56:22.306272 23468 tablet_service.cc:1467] Processing CreateTablet for tablet bb0cb22f98ef4d4cae5456d5880031b9 (DEFAULT_TABLE table=test-table [id=1935f6f06c68476793a1b61f97de4723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:56:22.307446 23468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb0cb22f98ef4d4cae5456d5880031b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:56:22.307451 23600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb0cb22f98ef4d4cae5456d5880031b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:56:22.316071 23680 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Bootstrap starting.
I20260620 14:56:22.318113 23680 tablet_bootstrap.cc:654] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Neither blocks nor log segments found. Creating new log.
I20260620 14:56:22.318784 23679 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Bootstrap starting.
I20260620 14:56:22.319260 23680 log.cc:824] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Log is configured to *not* fsync() on all Append() calls
I20260620 14:56:22.321177 23679 tablet_bootstrap.cc:654] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Neither blocks nor log segments found. Creating new log.
I20260620 14:56:22.321399 23680 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: No bootstrap required, opened a new log
I20260620 14:56:22.321586 23680 ts_tablet_manager.cc:1397] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260620 14:56:22.321914 23679 log.cc:824] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Log is configured to *not* fsync() on all Append() calls
I20260620 14:56:22.323472 23679 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: No bootstrap required, opened a new log
I20260620 14:56:22.323642 23679 ts_tablet_manager.cc:1397] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 14:56:22.324550 23680 raft_consensus.cc:348] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:22.324786 23680 raft_consensus.cc:374] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:56:22.324868 23680 raft_consensus.cc:729] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a114f16bd2d48fc9134a99ce48f8acc, State: Initialized, Role: FOLLOWER
I20260620 14:56:22.325292 23680 consensus_queue.cc:260] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:22.325975 23666 heartbeater.cc:503] Master 127.7.24.254:40807 was elected leader, sending a full tablet report...
I20260620 14:56:22.326259 23680 ts_tablet_manager.cc:1428] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260620 14:56:22.326737 23679 raft_consensus.cc:348] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:22.326961 23679 raft_consensus.cc:374] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:56:22.327018 23679 raft_consensus.cc:729] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fefd3b8f93e435c92da1cf57917994c, State: Initialized, Role: FOLLOWER
I20260620 14:56:22.327399 23679 consensus_queue.cc:260] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:22.328079 23534 heartbeater.cc:503] Master 127.7.24.254:40807 was elected leader, sending a full tablet report...
I20260620 14:56:22.328365 23679 ts_tablet_manager.cc:1428] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260620 14:56:22.348402 23397 tablet.cc:2369] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:56:22.446798 23683 raft_consensus.cc:482] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:56:22.447055 23683 raft_consensus.cc:504] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:22.448690 23683 leader_election.cc:288] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fefd3b8f93e435c92da1cf57917994c (127.7.24.194:36979), d5a59927975948d196f96d42b777952e (127.7.24.193:35421)
I20260620 14:56:22.452303 23677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.448758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:60503 (local address 127.7.24.194:36979)
0620 14:56:22.448968 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:22.448984 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:22.449189 (+ 205us) server_negotiation.cc:407] Connection header received
0620 14:56:22.449277 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:22.449279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:22.449323 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:22.449426 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:56:22.449732 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.450212 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.451306 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.451710 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.451841 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.451894 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:22.452080 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 14:56:22.452156 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260620 14:56:22.452324 23685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.448732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:52149 (local address 127.7.24.193:35421)
0620 14:56:22.449055 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:22.449059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:22.449074 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:56:22.449133 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:22.449139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:22.449212 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:22.449318 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:56:22.449621 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.450136 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.451301 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.451750 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.451892 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.452051 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:22.452093 (+ 42us) server_negotiation.cc:299] Negotiation successful
0620 14:56:22.452177 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":77,"threads_started":1}
I20260620 14:56:22.452323 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.448559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:35421 (local address 127.7.24.195:52149)
0620 14:56:22.448695 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:22.448721 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:22.448814 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:22.449321 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:22.449323 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:22.449336 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:56:22.449487 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.449494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.450255 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.450260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.451166 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.451175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.451879 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.451883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.451995 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.452015 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 14:56:22.452082 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 14:56:22.452193 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":52}
I20260620 14:56:22.452346 23686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.448559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:36979 (local address 127.7.24.195:60503)
0620 14:56:22.449091 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:22.449104 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:22.449196 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:22.449438 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:22.449441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:22.449459 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:56:22.449612 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.449618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.450327 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.450332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.451178 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:22.451186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.451837 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.451841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:22.451943 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.451972 (+ 29us) client_negotiation.cc:769] Sending connection context
0620 14:56:22.452064 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 14:56:22.452162 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":302,"thread_start_us":95,"threads_started":1}
I20260620 14:56:22.453002 23350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0cb22f98ef4d4cae5456d5880031b9" candidate_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a59927975948d196f96d42b777952e" is_pre_election: true
I20260620 14:56:22.453130 23488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0cb22f98ef4d4cae5456d5880031b9" candidate_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fefd3b8f93e435c92da1cf57917994c" is_pre_election: true
I20260620 14:56:22.453333 23350 raft_consensus.cc:2454] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a114f16bd2d48fc9134a99ce48f8acc in term 0.
I20260620 14:56:22.453344 23488 raft_consensus.cc:2454] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a114f16bd2d48fc9134a99ce48f8acc in term 0.
I20260620 14:56:22.453822 23553 leader_election.cc:302] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a114f16bd2d48fc9134a99ce48f8acc, 6fefd3b8f93e435c92da1cf57917994c; no voters:
I20260620 14:56:22.454077 23683 raft_consensus.cc:2781] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:56:22.454185 23683 raft_consensus.cc:482] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:56:22.454246 23683 raft_consensus.cc:3037] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:56:22.455317 23683 raft_consensus.cc:504] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:22.455694 23683 leader_election.cc:288] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [CANDIDATE]: Term 1 election: Requested vote from peers 6fefd3b8f93e435c92da1cf57917994c (127.7.24.194:36979), d5a59927975948d196f96d42b777952e (127.7.24.193:35421)
I20260620 14:56:22.456025 23350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0cb22f98ef4d4cae5456d5880031b9" candidate_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a59927975948d196f96d42b777952e"
I20260620 14:56:22.456067 23488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0cb22f98ef4d4cae5456d5880031b9" candidate_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fefd3b8f93e435c92da1cf57917994c"
I20260620 14:56:22.456167 23350 raft_consensus.cc:3037] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:56:22.456188 23488 raft_consensus.cc:3037] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:56:22.457125 23350 raft_consensus.cc:2454] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a114f16bd2d48fc9134a99ce48f8acc in term 1.
I20260620 14:56:22.457288 23488 raft_consensus.cc:2454] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a114f16bd2d48fc9134a99ce48f8acc in term 1.
I20260620 14:56:22.457402 23555 leader_election.cc:302] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a114f16bd2d48fc9134a99ce48f8acc, d5a59927975948d196f96d42b777952e; no voters:
I20260620 14:56:22.457581 23683 raft_consensus.cc:2781] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:56:22.457758 23683 raft_consensus.cc:686] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 1 LEADER]: Becoming Leader. State: Replica: 6a114f16bd2d48fc9134a99ce48f8acc, State: Running, Role: LEADER
I20260620 14:56:22.458078 23683 consensus_queue.cc:237] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:22.461412 23206 catalog_manager.cc:5495] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195). New cstate: current_term: 1 leader_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } health_report { overall_health: UNKNOWN } } }
I20260620 14:56:22.493772 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.490102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45780 (local address 127.7.24.195:39543)
0620 14:56:22.490261 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:22.490266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:22.490282 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:56:22.490400 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:22.490403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:22.490495 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:22.490612 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:56:22.490847 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.491415 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.492290 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:22.492524 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:22.492635 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:22.492938 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:56:22.493321 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:56:22.493457 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:22.493549 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 14:56:22.493603 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260620 14:56:22.496268 23667 tablet.cc:2369] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:56:22.499545 23350 raft_consensus.cc:1261] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Refusing update from remote peer 6a114f16bd2d48fc9134a99ce48f8acc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:56:22.499549 23488 raft_consensus.cc:1261] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Refusing update from remote peer 6a114f16bd2d48fc9134a99ce48f8acc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:56:22.500211 23687 consensus_queue.cc:1035] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:56:22.500427 23683 consensus_queue.cc:1035] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:56:22.507411 23692 mvcc.cc:205] Tried to move back new op lower bound from 7298938398711373824 to 7298938398551207936. Current Snapshot: MvccSnapshot[applied={T|T < 7298938398711373824}]
I20260620 14:56:22.508072 23690 mvcc.cc:205] Tried to move back new op lower bound from 7298938398711373824 to 7298938398551207936. Current Snapshot: MvccSnapshot[applied={T|T < 7298938398711373824}]
I20260620 14:56:22.508132 23691 mvcc.cc:205] Tried to move back new op lower bound from 7298938398711373824 to 7298938398551207936. Current Snapshot: MvccSnapshot[applied={T|T < 7298938398711373824}]
I20260620 14:56:22.517185 23208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:56:22.517374 23208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:56:22.519907 23208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Sending DeleteTablet for 3 replicas of tablet bb0cb22f98ef4d4cae5456d5880031b9
I20260620 14:56:22.520581 23600 tablet_service.cc:1514] Processing DeleteTablet for tablet bb0cb22f98ef4d4cae5456d5880031b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:56:22 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:45768
I20260620 14:56:22.520705 23468 tablet_service.cc:1514] Processing DeleteTablet for tablet bb0cb22f98ef4d4cae5456d5880031b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:56:22 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:41272
I20260620 14:56:22.520788 23330 tablet_service.cc:1514] Processing DeleteTablet for tablet bb0cb22f98ef4d4cae5456d5880031b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:56:22 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:45808
I20260620 14:56:22.520865 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 23246
I20260620 14:56:22.521839 23700 tablet_replica.cc:330] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: stopping tablet replica
I20260620 14:56:22.522140 23700 raft_consensus.cc:2229] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:56:22.522571 23700 raft_consensus.cc:2258] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:56:22.523867 23700 ts_tablet_manager.cc:1905] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:56:22.526180 23700 ts_tablet_manager.cc:1918] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:56:22.526285 23700 log.cc:1196] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/bb0cb22f98ef4d4cae5456d5880031b9
I20260620 14:56:22.526499 23701 tablet_replica.cc:330] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: stopping tablet replica
I20260620 14:56:22.526707 23700 ts_tablet_manager.cc:1939] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Deleting consensus metadata
I20260620 14:56:22.526971 23701 raft_consensus.cc:2229] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:56:22.527326 23701 raft_consensus.cc:2258] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:56:22.527873 23192 catalog_manager.cc:4841] TS 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195:39543): tablet bb0cb22f98ef4d4cae5456d5880031b9 (table test-table [id=1935f6f06c68476793a1b61f97de4723]) successfully deleted
W20260620 14:56:22.528077 23194 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:35421 (error 108)
I20260620 14:56:22.528930 23701 ts_tablet_manager.cc:1905] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 14:56:22.529264 23676 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:56:22.528952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:35421 (local address 127.0.0.1:45818)
0620 14:56:22.529113 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:22.529185 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:35421: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":76}
I20260620 14:56:22.529418 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 23400
W20260620 14:56:22.530591 23194 catalog_manager.cc:4584] TS d5a59927975948d196f96d42b777952e (127.7.24.193:35421): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bb0cb22f98ef4d4cae5456d5880031b9: Network error: Client connection negotiation failed: client connection to 127.7.24.193:35421: connect: Connection reset by peer (error 104)
I20260620 14:56:22.535866 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 23537
I20260620 14:56:22.537626 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:22.535981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:36979 (local address 127.0.0.1:41286)
0620 14:56:22.537510 (+ 1529us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:22.537559 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.194:36979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1454}
I20260620 14:56:22.541862 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 23175
I20260620 14:56:42.548506 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40807
--webserver_interface=127.7.24.254
--webserver_port=38021
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:56:42.658399 23703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:42.658694 23703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:42.658737 23703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:42.662809 23703 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 14:56:42.662869 23703 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 14:56:42.662884 23703 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 14:56:42.662899 23703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:56:42.662958 23703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:42.662997 23703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:42.663029 23703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:56:42.663064 23703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:56:42.668406 23703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=38021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:42.669509 23703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:42.670583 23703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:42.679409 23708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:42.679481 23711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:42.679502 23703 server_base.cc:1034] running on GCE node
W20260620 14:56:42.679409 23709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:42.680249 23703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:42.681339 23703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:42.682547 23703 hybrid_clock.cc:648] HybridClock initialized: now 1781967402682534 us; error 28 us; skew 500 ppm
Jun 20 14:56:42 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967402, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:42.685436 23703 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:56:42.686298 23703 webserver.cc:460] Webserver started at http://127.7.24.254:38021/ using document root <none> and password file <none>
I20260620 14:56:42.686877 23703 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:42.686937 23703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:42.691164 23703 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:56:42.693418 23718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:42.695032 23703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:56:42.695199 23703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "85d01da39508411cb57cc674a99db034"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "ea45a129d3fc3304a1d3d847f3284267"
server_key_iv: "104e759ff72f1db40225e2bd0ede3bc7"
server_key_version: "encryptionkey@0"
I20260620 14:56:42.696027 23703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:42.719383 23703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:42.723420 23703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:42.723644 23703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:42.731779 23703 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:40807
I20260620 14:56:42.732064 23770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:40807 every 8 connection(s)
I20260620 14:56:42.732954 23703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:56:42.735224 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 23703
I20260620 14:56:42.735682 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:35421
--local_ip_for_outbound_sockets=127.7.24.193
--tserver_master_addrs=127.7.24.254:40807
--webserver_port=34029
--webserver_interface=127.7.24.193
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:56:42.737725 23771 sys_catalog.cc:263] Verifying existing consensus state
I20260620 14:56:42.740447 23771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Bootstrap starting.
I20260620 14:56:42.765076 23771 log.cc:824] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Log is configured to *not* fsync() on all Append() calls
I20260620 14:56:42.779656 23771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:56:42.780092 23771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Bootstrap complete.
I20260620 14:56:42.784550 23771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:42.784946 23771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d01da39508411cb57cc674a99db034, State: Initialized, Role: FOLLOWER
I20260620 14:56:42.785450 23771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:42.785563 23771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:56:42.785650 23771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:56:42.785741 23771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 2 FOLLOWER]: Advancing to term 3
I20260620 14:56:42.787727 23771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:42.788074 23771 leader_election.cc:302] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85d01da39508411cb57cc674a99db034; no voters:
I20260620 14:56:42.788417 23771 leader_election.cc:288] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 14:56:42.788549 23775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 14:56:42.788798 23775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [term 3 LEADER]: Becoming Leader. State: Replica: 85d01da39508411cb57cc674a99db034, State: Running, Role: LEADER
I20260620 14:56:42.789103 23775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } }
I20260620 14:56:42.789635 23771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:56:42.790657 23777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85d01da39508411cb57cc674a99db034. Latest consensus state: current_term: 3 leader_uuid: "85d01da39508411cb57cc674a99db034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } } }
I20260620 14:56:42.790794 23777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: This master's current role is: LEADER
I20260620 14:56:42.791061 23776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "85d01da39508411cb57cc674a99db034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d01da39508411cb57cc674a99db034" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40807 } } }
I20260620 14:56:42.791138 23776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034 [sys.catalog]: This master's current role is: LEADER
I20260620 14:56:42.791154 23784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:56:42.796651 23784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:56:42.797398 23784 catalog_manager.cc:1259] Loaded cluster ID: 203cb7da662443a8a004ecf74ffc4fa7
I20260620 14:56:42.797459 23784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:56:42.799911 23784 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:56:42.800634 23784 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Loaded TSK: 0
I20260620 14:56:42.800715 23784 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 85d01da39508411cb57cc674a99db034: Loaded TSK: 1
F20260620 14:56:42.801433 23784 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff61c9279ad google::LogMessage::Fail() at ??:0
@ 0x7ff61c92a88a google::LogMessage::SendToLog() at ??:0
@ 0x7ff61c9274de google::LogMessage::Flush() at ??:0
@ 0x7ff61c928a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff61f5eb18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff61f5e3f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff62728c7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff62726699c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff627267178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7ff6272a8df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ff6272c782b std::function<>::operator()() at ??:0
@ 0x7ff627266b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ff627267f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff627264579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ff6272a8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff61eb793d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff61eb79c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ff61eb7b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff61eb6a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff6201ae6db start_thread at ??:0
@ 0x7ff61b42561f clone at ??:0
W20260620 14:56:42.893213 23773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:42.893464 23773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:42.893508 23773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:42.898043 23773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:42.898095 23773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:42.898159 23773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:56:42.904124 23773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:35421
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=34029
--tserver_master_addrs=127.7.24.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:42.905409 23773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:42.906560 23773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:42.917120 23821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:42.917110 23819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:42.917105 23818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:42.917750 23773 server_base.cc:1034] running on GCE node
I20260620 14:56:42.918226 23773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:42.919070 23773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:42.920289 23773 hybrid_clock.cc:648] HybridClock initialized: now 1781967402920252 us; error 47 us; skew 500 ppm
Jun 20 14:56:42 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967402, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:42.923730 23773 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:56:42.924672 23773 webserver.cc:460] Webserver started at http://127.7.24.193:34029/ using document root <none> and password file <none>
I20260620 14:56:42.925271 23773 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:42.925364 23773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:42.929425 23773 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:56:42.931730 23828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:42.933161 23773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 14:56:42.933311 23773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "d5a59927975948d196f96d42b777952e"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "c1163efc2fa0673a36d3a5401a0458af"
server_key_iv: "096f71fbb8e387577e810e7b24d08b11"
server_key_version: "encryptionkey@0"
I20260620 14:56:42.933885 23773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:42.946950 23773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:42.950230 23773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:42.950443 23773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:42.951100 23773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:56:42.952224 23835 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 14:56:42.957814 23773 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 14:56:42.957921 23773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260620 14:56:42.958015 23773 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 14:56:42.959753 23773 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 14:56:42.959830 23773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260620 14:56:42.960121 23835 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: Bootstrap starting.
I20260620 14:56:42.970620 23773 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:35421
I20260620 14:56:42.970852 23942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:35421 every 8 connection(s)
I20260620 14:56:42.971637 23773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:56:42.974259 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 23773
I20260620 14:56:42.974789 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:36979
--local_ip_for_outbound_sockets=127.7.24.194
--tserver_master_addrs=127.7.24.254:40807
--webserver_port=44529
--webserver_interface=127.7.24.194
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:56:42.982301 23835 log.cc:824] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: Log is configured to *not* fsync() on all Append() calls
I20260620 14:56:42.991909 23835 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:56:42.992445 23835 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: Bootstrap complete.
I20260620 14:56:42.993901 23835 ts_tablet_manager.cc:1397] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: Time spent bootstrapping tablet: real 0.034s user 0.022s sys 0.008s
I20260620 14:56:42.997295 23835 raft_consensus.cc:348] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:42.997901 23835 raft_consensus.cc:729] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a59927975948d196f96d42b777952e, State: Initialized, Role: FOLLOWER
I20260620 14:56:42.998430 23835 consensus_queue.cc:260] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:42.999492 23835 ts_tablet_manager.cc:1428] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260620 14:56:43.054101 23945 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:56:42.973592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:37633)
0620 14:56:42.974649 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:42.974711 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:42.976229 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:43.053037 (+ 76808us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":131,"threads_started":1}
W20260620 14:56:43.055202 23943 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:40807 (0 consecutive failures): Network error: Failed to ping master at 127.7.24.254:40807: Client connection negotiation failed: client connection to 127.7.24.254:40807: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 14:56:43.056133 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:43.055813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:42911)
0620 14:56:43.055978 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:43.056054 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260620 14:56:43.057171 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:43.056937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:54981)
0620 14:56:43.057066 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:43.057115 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260620 14:56:43.057404 23943 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 14:56:43.089907 23946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:43.090215 23946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:43.090310 23946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:43.094993 23946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:43.095085 23946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:43.095182 23946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:56:43.100484 23946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:36979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=44529
--tserver_master_addrs=127.7.24.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:43.101565 23946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:43.102661 23946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:43.112753 23956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:43.112753 23954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:43.112759 23953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:43.113112 23946 server_base.cc:1034] running on GCE node
I20260620 14:56:43.113549 23946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:43.114168 23946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:43.115374 23946 hybrid_clock.cc:648] HybridClock initialized: now 1781967403115339 us; error 49 us; skew 500 ppm
Jun 20 14:56:43 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967403, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:43.118098 23946 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:56:43.119045 23946 webserver.cc:460] Webserver started at http://127.7.24.194:44529/ using document root <none> and password file <none>
I20260620 14:56:43.119643 23946 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:43.119735 23946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:43.123667 23946 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260620 14:56:43.125831 23963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:43.127166 23946 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 14:56:43.127342 23946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "6fefd3b8f93e435c92da1cf57917994c"
format_stamp: "Formatted at 2026-06-20 14:56:00 on dist-test-slave-sms9"
server_key: "3c6500fc4166bd0a87d7a329e02e1de7"
server_key_iv: "766dcb6e2924dee3292891eef194565f"
server_key_version: "encryptionkey@0"
I20260620 14:56:43.127893 23946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:43.143055 23946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:43.146214 23946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:43.146513 23946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:43.147146 23946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:56:43.148216 23970 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 14:56:43.152344 23946 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 14:56:43.152433 23946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260620 14:56:43.152520 23946 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 14:56:43.154217 23946 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 14:56:43.154345 23946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260620 14:56:43.154610 23970 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Bootstrap starting.
I20260620 14:56:43.160418 23970 log.cc:824] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Log is configured to *not* fsync() on all Append() calls
I20260620 14:56:43.165217 23946 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:36979
I20260620 14:56:43.165279 24078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:36979 every 8 connection(s)
I20260620 14:56:43.166065 23946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:56:43.168831 23970 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:56:43.169286 23970 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Bootstrap complete.
I20260620 14:56:43.169493 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:43.167891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:52025)
0620 14:56:43.168409 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:43.168558 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":135,"threads_started":1}
I20260620 14:56:43.169643 23970 ts_tablet_manager.cc:1397] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
W20260620 14:56:43.169965 24079 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:40807 (0 consecutive failures): Network error: Failed to ping master at 127.7.24.254:40807: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
I20260620 14:56:43.170707 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:43.170412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:38635)
0620 14:56:43.170595 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:43.170647 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260620 14:56:43.171468 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:43.171209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:40687)
0620 14:56:43.171372 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:43.171417 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260620 14:56:43.171687 24079 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 14:56:43.172047 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 23946
I20260620 14:56:43.172456 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:39543
--local_ip_for_outbound_sockets=127.7.24.195
--tserver_master_addrs=127.7.24.254:40807
--webserver_port=40679
--webserver_interface=127.7.24.195
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:56:43.173094 23970 raft_consensus.cc:348] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:43.173494 23970 raft_consensus.cc:729] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fefd3b8f93e435c92da1cf57917994c, State: Initialized, Role: FOLLOWER
I20260620 14:56:43.174069 23970 consensus_queue.cc:260] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:43.175189 23970 ts_tablet_manager.cc:1428] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260620 14:56:43.175432 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:43.175184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:51299)
0620 14:56:43.175312 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:43.175364 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260620 14:56:43.176070 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:43.175852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:35533)
0620 14:56:43.175959 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:43.176008 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260620 14:56:43.284664 24082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:43.284976 24082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:43.285053 24082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:43.289106 24082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:43.289193 24082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:43.289289 24082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:56:43.294636 24082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:39543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=40679
--tserver_master_addrs=127.7.24.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:43.295771 24082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:43.296777 24082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:43.306439 24091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:43.306449 24088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:43.306501 24089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:43.306537 24082 server_base.cc:1034] running on GCE node
I20260620 14:56:43.307161 24082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:43.307792 24082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:43.309018 24082 hybrid_clock.cc:648] HybridClock initialized: now 1781967403309006 us; error 30 us; skew 500 ppm
Jun 20 14:56:43 dist-test-slave-sms9 krb5kdc[22626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967403, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:43.311808 24082 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:56:43.312614 24082 webserver.cc:460] Webserver started at http://127.7.24.195:40679/ using document root <none> and password file <none>
I20260620 14:56:43.313136 24082 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:43.313194 24082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:43.317015 24082 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:56:43.319067 24098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:43.320247 24082 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:56:43.320348 24082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "6a114f16bd2d48fc9134a99ce48f8acc"
format_stamp: "Formatted at 2026-06-20 14:56:01 on dist-test-slave-sms9"
server_key: "c2bd359e2787fa3e5c7156989f181c0a"
server_key_iv: "d51ec9b404e772ca7d3be475bbef2a0e"
server_key_version: "encryptionkey@0"
I20260620 14:56:43.320852 24082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:43.347028 24082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:43.350420 24082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:43.350603 24082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:43.351142 24082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:56:43.351917 24082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:56:43.351969 24082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:43.352010 24082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:56:43.352025 24082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:43.360872 24082 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:39543
I20260620 14:56:43.360931 24211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:39543 every 8 connection(s)
I20260620 14:56:43.361853 24082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:56:43.365523 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:43.363898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:49073)
0620 14:56:43.364750 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:43.364857 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":512,"threads_started":1}
W20260620 14:56:43.366004 24212 heartbeater.cc:644] Failed to heartbeat to 127.7.24.254:40807 (0 consecutive failures): Network error: Failed to ping master at 127.7.24.254:40807: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
I20260620 14:56:43.366583 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:43.366348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:36439)
0620 14:56:43.366488 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:43.366532 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260620 14:56:43.367168 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:43.366992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:58661)
0620 14:56:43.367079 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:43.367123 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 14:56:43.367328 24212 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 14:56:43.369866 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 24082
I20260620 14:56:44.058849 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:44.058172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:55103)
0620 14:56:44.058625 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:44.058706 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":163,"threads_started":1}
I20260620 14:56:44.177366 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:44.176888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:34545)
0620 14:56:44.177205 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:44.177271 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":75,"threads_started":1}
I20260620 14:56:44.368458 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:44.367974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:55579)
0620 14:56:44.368247 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:44.368309 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":71,"threads_started":1}
I20260620 14:56:44.630321 24219 raft_consensus.cc:482] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:56:44.630596 24219 raft_consensus.cc:504] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:44.631747 24219 leader_election.cc:288] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195:39543), d5a59927975948d196f96d42b777952e (127.7.24.193:35421)
Jun 20 14:56:44 dist-test-slave-sms9 krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967403, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.195@KRBTEST.COM
Jun 20 14:56:44 dist-test-slave-sms9 krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967403, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.193@KRBTEST.COM
I20260620 14:56:44.648591 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:44.632017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:39543 (local address 127.7.24.194:46591)
0620 14:56:44.632392 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:44.632421 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:44.632898 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:44.634676 (+ 1778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:44.634679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:44.634874 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:56:44.635003 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:44.635010 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:44.636891 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:44.636897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:44.638396 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:44.638409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:44.639659 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:44.639662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:44.639910 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:44.640753 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:56:44.640774 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:56:44.642644 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:56:44.645801 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:44.645807 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:44.645822 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:44.646216 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:44.646642 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:44.646739 (+ 97us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:44.646742 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:44.646921 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:44.647592 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:56:44.647599 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:56:44.647821 (+ 222us) client_negotiation.cc:769] Sending connection context
0620 14:56:44.648030 (+ 209us) client_negotiation.cc:240] Negotiation successful
0620 14:56:44.648381 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":143,"spinlock_wait_cycles":10880}
I20260620 14:56:44.648591 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:44.632197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:46591 (local address 127.7.24.195:39543)
0620 14:56:44.632593 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:44.632617 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:44.632658 (+ 41us) server_negotiation.cc:407] Connection header received
0620 14:56:44.634082 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:44.634090 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:44.634420 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:44.634689 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:44.635140 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:44.636683 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:44.638647 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:44.639333 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:44.639474 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:44.642893 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:44.642900 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:44.642915 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:44.642957 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:44.645128 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:44.646336 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:44.646343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:44.646348 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:44.646547 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:44.647037 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:44.647042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:44.647044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:44.647471 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:44.647618 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:44.648136 (+ 518us) server_negotiation.cc:299] Negotiation successful
0620 14:56:44.648381 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":72,"threads_started":1}
I20260620 14:56:44.650080 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:44.632017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:35421 (local address 127.7.24.194:59985)
0620 14:56:44.632392 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:44.632421 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:44.632887 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:44.633906 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:44.633914 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:44.634272 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:56:44.634884 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:44.634904 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:44.636902 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:44.636908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:44.638399 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:44.638412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:44.639689 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:44.639692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:44.639907 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:44.644646 (+ 4739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:56:44.644661 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:56:44.646001 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:56:44.648249 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:44.648254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:44.648258 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:44.648483 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:44.648919 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:44.648923 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:44.648925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:44.649009 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:44.649693 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:56:44.649695 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:56:44.649771 (+ 76us) client_negotiation.cc:769] Sending connection context
0620 14:56:44.649847 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 14:56:44.649947 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":134,"spinlock_wait_cycles":15360,"thread_start_us":76,"threads_started":1}
I20260620 14:56:44.650483 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:44.632197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:59985 (local address 127.7.24.193:35421)
0620 14:56:44.632588 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:44.632596 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:44.632628 (+ 32us) server_negotiation.cc:407] Connection header received
0620 14:56:44.633323 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:44.633331 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:44.633596 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:44.633815 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:44.635111 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:44.636694 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:44.638662 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:44.639350 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:44.639474 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:44.646189 (+ 6715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:44.646196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:44.646212 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:44.646253 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:44.648121 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:44.648612 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:44.648618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:44.648623 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:44.648795 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:44.649116 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:44.649119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:44.649121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:44.649577 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:44.649714 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:44.650017 (+ 303us) server_negotiation.cc:299] Negotiation successful
0620 14:56:44.650294 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":69,"threads_started":1}
W20260620 14:56:44.651333 23964 leader_election.cc:341] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195:39543): Not found: Tablet not found: bb0cb22f98ef4d4cae5456d5880031b9
I20260620 14:56:44.651532 23897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0cb22f98ef4d4cae5456d5880031b9" candidate_uuid: "6fefd3b8f93e435c92da1cf57917994c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d5a59927975948d196f96d42b777952e" is_pre_election: true
I20260620 14:56:44.651875 23897 raft_consensus.cc:2454] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fefd3b8f93e435c92da1cf57917994c in term 1.
I20260620 14:56:44.652580 23966 leader_election.cc:302] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fefd3b8f93e435c92da1cf57917994c, d5a59927975948d196f96d42b777952e; no voters: 6a114f16bd2d48fc9134a99ce48f8acc
I20260620 14:56:44.652822 24219 raft_consensus.cc:2781] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 14:56:44.652904 24219 raft_consensus.cc:482] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:56:44.652954 24219 raft_consensus.cc:3037] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 1 FOLLOWER]: Advancing to term 2
I20260620 14:56:44.655118 24219 raft_consensus.cc:504] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:44.655488 24219 leader_election.cc:288] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [CANDIDATE]: Term 2 election: Requested vote from peers 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195:39543), d5a59927975948d196f96d42b777952e (127.7.24.193:35421)
I20260620 14:56:44.655869 23897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0cb22f98ef4d4cae5456d5880031b9" candidate_uuid: "6fefd3b8f93e435c92da1cf57917994c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d5a59927975948d196f96d42b777952e"
I20260620 14:56:44.656038 23897 raft_consensus.cc:3037] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 1 FOLLOWER]: Advancing to term 2
W20260620 14:56:44.656167 23964 leader_election.cc:341] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195:39543): Not found: Tablet not found: bb0cb22f98ef4d4cae5456d5880031b9
I20260620 14:56:44.658053 23897 raft_consensus.cc:2454] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fefd3b8f93e435c92da1cf57917994c in term 2.
I20260620 14:56:44.658340 23966 leader_election.cc:302] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fefd3b8f93e435c92da1cf57917994c, d5a59927975948d196f96d42b777952e; no voters: 6a114f16bd2d48fc9134a99ce48f8acc
I20260620 14:56:44.658601 24219 raft_consensus.cc:2781] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 2 FOLLOWER]: Leader election won for term 2
I20260620 14:56:44.658809 24219 raft_consensus.cc:686] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [term 2 LEADER]: Becoming Leader. State: Replica: 6fefd3b8f93e435c92da1cf57917994c, State: Running, Role: LEADER
I20260620 14:56:44.659170 24219 consensus_queue.cc:237] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:44.659572 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:44.659286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:57703)
0620 14:56:44.659421 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:44.659497 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 14:56:45.060362 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:45.059862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:36583)
0620 14:56:45.060147 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:45.060214 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":83,"threads_started":1}
W20260620 14:56:45.112777 23964 consensus_peers.cc:487] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c -> Peer 6a114f16bd2d48fc9134a99ce48f8acc (127.7.24.195:39543): Couldn't send request to peer 6a114f16bd2d48fc9134a99ce48f8acc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb0cb22f98ef4d4cae5456d5880031b9. This is attempt 1: this message will repeat every 5th retry.
I20260620 14:56:45.156872 23897 raft_consensus.cc:1261] T bb0cb22f98ef4d4cae5456d5880031b9 P d5a59927975948d196f96d42b777952e [term 2 FOLLOWER]: Refusing update from remote peer 6fefd3b8f93e435c92da1cf57917994c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260620 14:56:45.157743 24219 consensus_queue.cc:1035] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260620 14:56:45.160027 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:45.159730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:59663)
0620 14:56:45.159869 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:45.159936 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 14:56:45.162942 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:45.162541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:38353)
0620 14:56:45.162777 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:45.162839 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":72,"threads_started":1}
I20260620 14:56:45.162981 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:45.162730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:48543)
0620 14:56:45.162835 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:45.162884 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 14:56:45.369693 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:45.369267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:47803)
0620 14:56:45.369543 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:45.369604 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":72,"threads_started":1}
I20260620 14:56:45.645481 24233 ts_tablet_manager.cc:927] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Initiating tablet copy from peer 6fefd3b8f93e435c92da1cf57917994c (127.7.24.194:36979)
I20260620 14:56:45.646145 24233 tablet_copy_client.cc:316] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: tablet copy: Beginning tablet copy session from remote peer at address 127.7.24.194:36979
Jun 20 14:56:45 dist-test-slave-sms9 krb5kdc[22626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967403, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.194@KRBTEST.COM
I20260620 14:56:45.656347 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:45.646745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:36979 (local address 127.7.24.195:46163)
0620 14:56:45.646890 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:45.646911 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 14:56:45.647039 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:56:45.647736 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:56:45.647742 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:56:45.648109 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:56:45.648229 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:45.648235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:45.649231 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:45.649234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:45.649891 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:56:45.649901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:56:45.650622 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:56:45.650625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:56:45.650701 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:45.651278 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:56:45.651290 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:56:45.652664 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:56:45.654824 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:45.654833 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:45.654838 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:45.655063 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:45.655340 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:56:45.655342 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:56:45.655344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:56:45.655419 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:56:45.655853 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:56:45.655858 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:56:45.655963 (+ 105us) client_negotiation.cc:769] Sending connection context
0620 14:56:45.656078 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 14:56:45.656210 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 14:56:45.656427 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:45.646945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:46163 (local address 127.7.24.194:36979)
0620 14:56:45.647275 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:45.647284 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:45.647304 (+ 20us) server_negotiation.cc:407] Connection header received
0620 14:56:45.647348 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:45.647353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:45.647566 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:45.647689 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:45.648344 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:45.649115 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:45.650065 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:45.650503 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:45.650604 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:45.652826 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:45.652835 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:45.652846 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:45.652871 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:45.654718 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:45.655162 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:45.655167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:45.655168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:45.655259 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:45.655496 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:45.655498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:45.655499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:45.655746 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:45.655849 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:45.656162 (+ 313us) server_negotiation.cc:299] Negotiation successful
0620 14:56:45.656285 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":75,"threads_started":1}
I20260620 14:56:45.657612 24053 tablet_copy_service.cc:140] P 6fefd3b8f93e435c92da1cf57917994c: Received BeginTabletCopySession request for tablet bb0cb22f98ef4d4cae5456d5880031b9 from peer 6a114f16bd2d48fc9134a99ce48f8acc ({username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:46163)
I20260620 14:56:45.657768 24053 tablet_copy_service.cc:161] P 6fefd3b8f93e435c92da1cf57917994c: Beginning new tablet copy session on tablet bb0cb22f98ef4d4cae5456d5880031b9 from peer 6a114f16bd2d48fc9134a99ce48f8acc at {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:46163: session id = 6a114f16bd2d48fc9134a99ce48f8acc-bb0cb22f98ef4d4cae5456d5880031b9
I20260620 14:56:45.659066 24053 tablet_copy_source_session.cc:205] T bb0cb22f98ef4d4cae5456d5880031b9 P 6fefd3b8f93e435c92da1cf57917994c: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 14:56:45.663194 24233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb0cb22f98ef4d4cae5456d5880031b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:56:45.669596 24233 tablet_copy_client.cc:791] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: tablet copy: Starting download of 0 data blocks...
I20260620 14:56:45.669790 24233 tablet_copy_client.cc:655] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: tablet copy: Starting download of 1 WAL segments...
I20260620 14:56:45.671687 24233 tablet_copy_client.cc:531] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 14:56:45.673317 24233 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Bootstrap starting.
I20260620 14:56:45.678828 24233 log.cc:824] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Log is configured to *not* fsync() on all Append() calls
I20260620 14:56:45.685463 24233 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 14:56:45.685808 24233 tablet_bootstrap.cc:492] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Bootstrap complete.
I20260620 14:56:45.686151 24233 ts_tablet_manager.cc:1397] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260620 14:56:45.688508 24233 raft_consensus.cc:348] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:45.688752 24233 raft_consensus.cc:729] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a114f16bd2d48fc9134a99ce48f8acc, State: Initialized, Role: FOLLOWER
I20260620 14:56:45.689225 24233 consensus_queue.cc:260] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a114f16bd2d48fc9134a99ce48f8acc" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39543 } } peers { permanent_uuid: "6fefd3b8f93e435c92da1cf57917994c" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 36979 } } peers { permanent_uuid: "d5a59927975948d196f96d42b777952e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 35421 } }
I20260620 14:56:45.690160 24233 ts_tablet_manager.cc:1428] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20260620 14:56:45.690644 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:45.690291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:35485)
0620 14:56:45.690458 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:45.690543 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260620 14:56:45.690943 24053 tablet_copy_service.cc:342] P 6fefd3b8f93e435c92da1cf57917994c: Request end of tablet copy session 6a114f16bd2d48fc9134a99ce48f8acc-bb0cb22f98ef4d4cae5456d5880031b9 received from {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:46163
I20260620 14:56:45.691075 24053 tablet_copy_service.cc:433] P 6fefd3b8f93e435c92da1cf57917994c: ending tablet copy session 6a114f16bd2d48fc9134a99ce48f8acc-bb0cb22f98ef4d4cae5456d5880031b9 on tablet bb0cb22f98ef4d4cae5456d5880031b9 with peer 6a114f16bd2d48fc9134a99ce48f8acc
I20260620 14:56:45.691323 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:45.691107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:40793)
0620 14:56:45.691212 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:45.691261 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260620 14:56:46.164151 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:46.163730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:35829)
0620 14:56:46.163991 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:46.164051 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
I20260620 14:56:46.164183 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:46.163732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:40995)
0620 14:56:46.164048 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:46.164097 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":91,"threads_started":1}
I20260620 14:56:46.177736 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:46.177445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:38257)
0620 14:56:46.177582 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:46.177655 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 14:56:46.178027 24166 raft_consensus.cc:1203] T bb0cb22f98ef4d4cae5456d5880031b9 P 6a114f16bd2d48fc9134a99ce48f8acc [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260620 14:56:46.179015 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:46.178781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:45575)
0620 14:56:46.178895 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:46.178951 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 14:56:47.165441 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:47.164930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:47685)
0620 14:56:47.165229 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:47.165302 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":79,"threads_started":1}
I20260620 14:56:47.178951 24244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:47.178553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:39037)
0620 14:56:47.178803 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:47.178854 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":95,"threads_started":1}
I20260620 14:56:47.180136 24245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:47.179708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:39203)
0620 14:56:47.180001 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:47.180047 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":57,"threads_started":1}
I20260620 14:56:48.167104 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:48.166651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:34683)
0620 14:56:48.166963 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:48.167017 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":107,"threads_started":1}
I20260620 14:56:48.180351 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:48.179875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:43681)
0620 14:56:48.180141 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:48.180202 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":108,"threads_started":1}
I20260620 14:56:48.181098 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:48.180887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:51449)
0620 14:56:48.181001 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:48.181046 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260620 14:56:48.181324 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:48.180887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:37739)
0620 14:56:48.181154 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:48.181221 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":77,"threads_started":1}
I20260620 14:56:49.168394 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:49.167939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:51513)
0620 14:56:49.168231 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:49.168297 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":94,"threads_started":1}
I20260620 14:56:49.182314 24250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:49.181846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:39217)
0620 14:56:49.182163 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:49.182219 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":127,"threads_started":1}
I20260620 14:56:49.182641 24251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:49.182192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:57125)
0620 14:56:49.182457 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:49.182540 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
I20260620 14:56:50.169721 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:50.169279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:38751)
0620 14:56:50.169559 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:50.169622 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":75,"threads_started":1}
I20260620 14:56:50.183738 24253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:50.183247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:52101)
0620 14:56:50.183571 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:50.183636 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":105,"threads_started":1}
I20260620 14:56:50.183765 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:50.183320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:35001)
0620 14:56:50.183617 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:50.183677 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":59,"threads_started":1}
I20260620 14:56:51.171154 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:51.170738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:43401)
0620 14:56:51.171005 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:51.171063 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":77,"threads_started":1}
I20260620 14:56:51.184916 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:51.184518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:42981)
0620 14:56:51.184766 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:51.184820 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":95,"threads_started":1}
I20260620 14:56:51.184991 24257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:51.184618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:47267)
0620 14:56:51.184859 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:51.184908 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":94,"threads_started":1}
I20260620 14:56:52.172474 24258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:52.172003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:35877)
0620 14:56:52.172286 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:52.172365 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":71,"threads_started":1}
I20260620 14:56:52.186268 24260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:52.185765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:44837)
0620 14:56:52.186094 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:52.186154 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":89,"threads_started":1}
I20260620 14:56:52.186269 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:52.185774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:37809)
0620 14:56:52.186094 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:52.186154 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":83,"threads_started":1}
I20260620 14:56:53.173862 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:53.173421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:42859)
0620 14:56:53.173711 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:53.173768 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":74,"threads_started":1}
I20260620 14:56:53.187601 24262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:53.187123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:50337)
0620 14:56:53.187420 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:53.187503 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":111,"threads_started":1}
I20260620 14:56:53.187623 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:53.187174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:55823)
0620 14:56:53.187485 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:53.187538 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":61,"threads_started":1}
I20260620 14:56:54.175179 24264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:54.174697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:56871)
0620 14:56:54.175018 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:54.175081 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":81,"threads_started":1}
I20260620 14:56:54.188938 24265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:54.188446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:60869)
0620 14:56:54.188788 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:54.188840 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":82,"threads_started":1}
I20260620 14:56:54.189007 24266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:54.188522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:58745)
0620 14:56:54.188868 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:54.188922 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":86,"threads_started":1}
I20260620 14:56:55.176532 24267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:55.176099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:59447)
0620 14:56:55.176362 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:55.176431 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":77,"threads_started":1}
I20260620 14:56:55.190177 24268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:55.189777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:39227)
0620 14:56:55.190032 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:55.190083 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":99,"threads_started":1}
I20260620 14:56:55.190208 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:55.189841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:37883)
0620 14:56:55.190074 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:55.190123 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":90,"threads_started":1}
I20260620 14:56:56.177837 24270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:56.177426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:38059)
0620 14:56:56.177681 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:56.177742 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
I20260620 14:56:56.191531 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:56.191021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:41473)
0620 14:56:56.191361 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:56.191434 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":72,"threads_started":1}
I20260620 14:56:56.191560 24272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:56.191159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:54695)
0620 14:56:56.191414 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:56.191472 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":95,"threads_started":1}
I20260620 14:56:57.179095 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:57.178665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:51437)
0620 14:56:57.178938 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:57.178999 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":85,"threads_started":1}
I20260620 14:56:57.192754 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:57.192288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:46619)
0620 14:56:57.192596 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:57.192658 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":73,"threads_started":1}
I20260620 14:56:57.192763 24275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:57.192288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:55485)
0620 14:56:57.192612 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:57.192665 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":106,"threads_started":1}
I20260620 14:56:58.180733 24277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:58.180144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.193:49361)
0620 14:56:58.180549 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:58.180617 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":102,"threads_started":1}
I20260620 14:56:58.193965 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:58.193529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.195:46271)
0620 14:56:58.193780 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:58.193843 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":68,"threads_started":1}
I20260620 14:56:58.194239 24279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:58.193823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40807 (local address 127.7.24.194:44463)
0620 14:56:58.194062 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0620 14:56:58.194136 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":69,"threads_started":1}
I20260620 14:56:58.377904 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 23773
I20260620 14:56:58.383744 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 23946
I20260620 14:56:58.389304 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 24082
2026-06-20T14:56:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59478 ms)
[----------] 30 tests from SecurityITest (241723 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:56:58 dist-test-slave-sms9 krb5kdc[24282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:56:58 dist-test-slave-sms9 krb5kdc[24282](info): set up 2 sockets
Jun 20 14:56:58 dist-test-slave-sms9 krb5kdc[24282](info): commencing operation
krb5kdc: starting...
W20260620 14:56:59.454217 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:56:59 dist-test-slave-sms9 krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:56:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:56:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:56:59.577383 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:44641
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:44641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:56:59.686792 24295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:59.687112 24295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:59.687193 24295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:59.691200 24295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:56:59.691313 24295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:59.691350 24295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:59.691411 24295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:56:59.691468 24295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:56:59.696851 24295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:44641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:44641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:59.697952 24295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:59.698995 24295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:56:59.707901 24295 server_base.cc:1034] running on GCE node
W20260620 14:56:59.707835 24301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:59.707835 24300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:59.707840 24303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:59.708695 24295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:59.709828 24295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:59.710999 24295 hybrid_clock.cc:648] HybridClock initialized: now 1781967419710965 us; error 46 us; skew 500 ppm
Jun 20 14:56:59 dist-test-slave-sms9 krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967419, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:59.713838 24295 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:56:59.714829 24295 webserver.cc:460] Webserver started at http://127.7.24.254:43813/ using document root <none> and password file <none>
I20260620 14:56:59.715390 24295 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:59.715464 24295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:59.715636 24295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:56:59.717653 24295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "df43ad3d2d434953897c8fa082f963af"
format_stamp: "Formatted at 2026-06-20 14:56:59 on dist-test-slave-sms9"
server_key: "aee7bf4ee96ef2441701c5166c086d3e"
server_key_iv: "8ba23b925a006173693923cca8cf598b"
server_key_version: "encryptionkey@0"
I20260620 14:56:59.718171 24295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "df43ad3d2d434953897c8fa082f963af"
format_stamp: "Formatted at 2026-06-20 14:56:59 on dist-test-slave-sms9"
server_key: "aee7bf4ee96ef2441701c5166c086d3e"
server_key_iv: "8ba23b925a006173693923cca8cf598b"
server_key_version: "encryptionkey@0"
I20260620 14:56:59.721915 24295 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:56:59.724574 24310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:59.725980 24295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:56:59.726099 24295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "df43ad3d2d434953897c8fa082f963af"
format_stamp: "Formatted at 2026-06-20 14:56:59 on dist-test-slave-sms9"
server_key: "aee7bf4ee96ef2441701c5166c086d3e"
server_key_iv: "8ba23b925a006173693923cca8cf598b"
server_key_version: "encryptionkey@0"
I20260620 14:56:59.726205 24295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:59.735445 24295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:56:59.738230 24295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:56:59.738384 24295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:56:59.746109 24295 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:44641
I20260620 14:56:59.746134 24362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:44641 every 8 connection(s)
I20260620 14:56:59.747225 24295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:56:59.750027 24363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:56:59.754104 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 24295
I20260620 14:56:59.754257 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:56:59.754588 7267 external_mini_cluster.cc:1428] Setting key 84cd9564c344d86e3d2bef3c46224714
I20260620 14:56:59.756333 24363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af: Bootstrap starting.
I20260620 14:56:59.758641 24363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af: Neither blocks nor log segments found. Creating new log.
I20260620 14:56:59.759411 24363 log.cc:824] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af: Log is configured to *not* fsync() on all Append() calls
I20260620 14:56:59.761381 24363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af: No bootstrap required, opened a new log
Jun 20 14:56:59 dist-test-slave-sms9 krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:56:59.765741 24363 raft_consensus.cc:348] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df43ad3d2d434953897c8fa082f963af" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44641 } }
I20260620 14:56:59.766072 24363 raft_consensus.cc:374] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:56:59.766165 24363 raft_consensus.cc:729] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df43ad3d2d434953897c8fa082f963af, State: Initialized, Role: FOLLOWER
I20260620 14:56:59.766672 24363 consensus_queue.cc:260] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df43ad3d2d434953897c8fa082f963af" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44641 } }
I20260620 14:56:59.766799 24363 raft_consensus.cc:388] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:56:59.766889 24363 raft_consensus.cc:482] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:56:59.766990 24363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:56:59.768209 24363 raft_consensus.cc:504] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df43ad3d2d434953897c8fa082f963af" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44641 } }
I20260620 14:56:59.768564 24363 leader_election.cc:302] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df43ad3d2d434953897c8fa082f963af; no voters:
I20260620 14:56:59.768955 24363 leader_election.cc:288] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:56:59.769084 24368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:56:59.769309 24368 raft_consensus.cc:686] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [term 1 LEADER]: Becoming Leader. State: Replica: df43ad3d2d434953897c8fa082f963af, State: Running, Role: LEADER
I20260620 14:56:59.769694 24368 consensus_queue.cc:237] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df43ad3d2d434953897c8fa082f963af" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44641 } }
I20260620 14:56:59.770335 24363 sys_catalog.cc:564] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:56:59.770879 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:56:59.756125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58954 (local address 127.7.24.254:44641)
0620 14:56:59.756646 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0620 14:56:59.756662 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 14:56:59.756707 (+ 45us) server_negotiation.cc:407] Connection header received
0620 14:56:59.757400 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:56:59.757420 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:56:59.757747 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:56:59.758100 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:56:59.759042 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:59.760317 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:59.761241 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:56:59.761782 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:56:59.761965 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:56:59.764901 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:56:59.764907 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:56:59.764921 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:56:59.764950 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:56:59.767164 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:59.767687 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:59.767692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:59.767696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:59.767821 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:56:59.768138 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:56:59.768141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:56:59.768142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:56:59.768712 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:56:59.768884 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 14:56:59.769572 (+ 688us) server_negotiation.cc:299] Negotiation successful
0620 14:56:59.769890 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":143,"threads_started":1}
I20260620 14:56:59.771602 24369 sys_catalog.cc:455] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df43ad3d2d434953897c8fa082f963af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df43ad3d2d434953897c8fa082f963af" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44641 } } }
I20260620 14:56:59.771741 24369 sys_catalog.cc:458] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [sys.catalog]: This master's current role is: LEADER
I20260620 14:56:59.772161 24370 sys_catalog.cc:455] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [sys.catalog]: SysCatalogTable state changed. Reason: New leader df43ad3d2d434953897c8fa082f963af. Latest consensus state: current_term: 1 leader_uuid: "df43ad3d2d434953897c8fa082f963af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df43ad3d2d434953897c8fa082f963af" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 44641 } } }
I20260620 14:56:59.772384 24370 sys_catalog.cc:458] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af [sys.catalog]: This master's current role is: LEADER
I20260620 14:56:59.773252 24377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:56:59.775996 24377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:56:59.781304 24377 catalog_manager.cc:1347] Generated new cluster ID: 82c06f6222bc49c5b3e3bb0ab10f27e8
I20260620 14:56:59.781371 24377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:56:59.787528 24377 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:56:59.788687 24377 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:56:59.794176 24377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af: Generated new TSK 0
I20260620 14:56:59.794924 24377 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:56:59.823956 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44641
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:56:59.934552 24389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:56:59.934840 24389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:56:59.934921 24389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:56:59.939023 24389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:56:59.939107 24389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:56:59.939194 24389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:56:59.944568 24389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:56:59.945688 24389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:56:59.946771 24389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:56:59.956207 24397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:59.956204 24395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:56:59.956212 24394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:56:59.956542 24389 server_base.cc:1034] running on GCE node
I20260620 14:56:59.957036 24389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:56:59.957612 24389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:56:59.958798 24389 hybrid_clock.cc:648] HybridClock initialized: now 1781967419958780 us; error 32 us; skew 500 ppm
Jun 20 14:56:59 dist-test-slave-sms9 krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967419, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:56:59.961555 24389 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:56:59.962352 24389 webserver.cc:460] Webserver started at http://127.7.24.193:35529/ using document root <none> and password file <none>
I20260620 14:56:59.962934 24389 fs_manager.cc:359] Metadata directory not provided
I20260620 14:56:59.962996 24389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:56:59.963164 24389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:56:59.965204 24389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "b8c1b5d7974a482f9dbeedcd726a3228"
format_stamp: "Formatted at 2026-06-20 14:56:59 on dist-test-slave-sms9"
server_key: "4154d30cb9674e2642ccd88a2c3114c2"
server_key_iv: "a992ee79ac7374486659100e5ed45b18"
server_key_version: "encryptionkey@0"
I20260620 14:56:59.965706 24389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "b8c1b5d7974a482f9dbeedcd726a3228"
format_stamp: "Formatted at 2026-06-20 14:56:59 on dist-test-slave-sms9"
server_key: "4154d30cb9674e2642ccd88a2c3114c2"
server_key_iv: "a992ee79ac7374486659100e5ed45b18"
server_key_version: "encryptionkey@0"
I20260620 14:56:59.969290 24389 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 14:56:59.971830 24404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:56:59.973124 24389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:56:59.973250 24389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "b8c1b5d7974a482f9dbeedcd726a3228"
format_stamp: "Formatted at 2026-06-20 14:56:59 on dist-test-slave-sms9"
server_key: "4154d30cb9674e2642ccd88a2c3114c2"
server_key_iv: "a992ee79ac7374486659100e5ed45b18"
server_key_version: "encryptionkey@0"
I20260620 14:56:59.973356 24389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:56:59.997283 24389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:00.000976 24389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:00.001187 24389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:00.001793 24389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:00.002712 24389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:00.002763 24389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:00.002835 24389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:00.002867 24389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:00.011757 24389 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:44831
I20260620 14:57:00.011816 24517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:44831 every 8 connection(s)
I20260620 14:57:00.012678 24389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:57:00.021097 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 24389
I20260620 14:57:00.021211 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:57:00.021550 7267 external_mini_cluster.cc:1428] Setting key 6b7ef926934d640c68e6f2a0061b3ee8
Jun 20 14:57:00 dist-test-slave-sms9 krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967419, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:00.026616 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.014502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:50019 (local address 127.7.24.254:44641)
0620 14:57:00.014663 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:00.014667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:00.015601 (+ 934us) server_negotiation.cc:407] Connection header received
0620 14:57:00.016873 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:00.016876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:00.016945 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:00.017047 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:00.018187 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.018776 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.019898 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.020129 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.020243 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.022917 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:00.022921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:00.022924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:00.022955 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:00.024455 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:00.024975 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:00.024979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:00.024981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:00.025067 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:00.025409 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:00.025413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:00.025414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:00.025600 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:00.025709 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:00.026306 (+ 597us) server_negotiation.cc:299] Negotiation successful
0620 14:57:00.026414 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 14:57:00.027750 24520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.014964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:44641 (local address 127.7.24.193:50019)
0620 14:57:00.015447 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:00.015489 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:00.016659 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:00.017227 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:00.017233 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:00.017587 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:00.018038 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.018051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.018900 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.018903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.019767 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.019776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.020282 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.020286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.020463 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.021074 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:00.021091 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:00.022752 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:00.024592 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:00.024598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:00.024612 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:00.024863 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:00.025167 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:00.025170 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:00.025172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:00.025312 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:00.025708 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:00.025714 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:00.026041 (+ 327us) client_negotiation.cc:769] Sending connection context
0620 14:57:00.026251 (+ 210us) client_negotiation.cc:240] Negotiation successful
0620 14:57:00.026628 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":151,"threads_started":1}
I20260620 14:57:00.029052 24518 heartbeater.cc:347] Connected to a master server at 127.7.24.254:44641
I20260620 14:57:00.029320 24518 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:00.029955 24518 heartbeater.cc:511] Master 127.7.24.254:44641 requested a full tablet report, sending...
I20260620 14:57:00.031510 24325 ts_manager.cc:194] Registered new tserver with Master: b8c1b5d7974a482f9dbeedcd726a3228 (127.7.24.193:44831)
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:57:00.032749 24325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:50019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:57:00.047384 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44641
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:00.159667 24523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:00.159945 24523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:00.159986 24523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:00.164215 24523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:00.164287 24523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:00.164423 24523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:57:00.169821 24523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:00.171131 24523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:00.172259 24523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:00.182124 24531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:00.182075 24529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:00.182070 24528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:00.182512 24523 server_base.cc:1034] running on GCE node
I20260620 14:57:00.182917 24523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:00.183578 24523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:00.184778 24523 hybrid_clock.cc:648] HybridClock initialized: now 1781967420184761 us; error 29 us; skew 500 ppm
Jun 20 14:57:00 dist-test-slave-sms9 krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967420, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:00.187556 24523 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:57:00.188450 24523 webserver.cc:460] Webserver started at http://127.7.24.194:39011/ using document root <none> and password file <none>
I20260620 14:57:00.189038 24523 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:00.189131 24523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:00.189358 24523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:00.191460 24523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "c97ceb842531433db4af349bdeddc52b"
format_stamp: "Formatted at 2026-06-20 14:57:00 on dist-test-slave-sms9"
server_key: "3a885d19dc7645e846ae4296682e4be5"
server_key_iv: "e99114c8ac58dfabad85224d20758154"
server_key_version: "encryptionkey@0"
I20260620 14:57:00.192028 24523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "c97ceb842531433db4af349bdeddc52b"
format_stamp: "Formatted at 2026-06-20 14:57:00 on dist-test-slave-sms9"
server_key: "3a885d19dc7645e846ae4296682e4be5"
server_key_iv: "e99114c8ac58dfabad85224d20758154"
server_key_version: "encryptionkey@0"
I20260620 14:57:00.195698 24523 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:57:00.198097 24538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:00.199364 24523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:00.199509 24523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "c97ceb842531433db4af349bdeddc52b"
format_stamp: "Formatted at 2026-06-20 14:57:00 on dist-test-slave-sms9"
server_key: "3a885d19dc7645e846ae4296682e4be5"
server_key_iv: "e99114c8ac58dfabad85224d20758154"
server_key_version: "encryptionkey@0"
I20260620 14:57:00.199620 24523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:00.218607 24523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:00.221910 24523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:00.222219 24523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:00.222872 24523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:00.223711 24523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:00.223781 24523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:00.223852 24523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:00.223894 24523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:00.233314 24523 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:44529
I20260620 14:57:00.233345 24651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:44529 every 8 connection(s)
I20260620 14:57:00.234231 24523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
Jun 20 14:57:00 dist-test-slave-sms9 krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967420, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:00.244024 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 24523
I20260620 14:57:00.244151 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:57:00.244445 7267 external_mini_cluster.cc:1428] Setting key 10a27733f65c6fc26c8468bc420461cf
I20260620 14:57:00.249017 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.236318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:46153 (local address 127.7.24.254:44641)
0620 14:57:00.236506 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:00.236509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:00.237416 (+ 907us) server_negotiation.cc:407] Connection header received
0620 14:57:00.238338 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:00.238342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:00.238402 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:00.238502 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:00.239673 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.240216 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.241386 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.241632 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.241747 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.244639 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:00.244642 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:00.244644 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:00.244672 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:00.246345 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:00.247094 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:00.247096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:00.247098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:00.247177 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:00.247645 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:00.247648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:00.247649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:00.247969 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:00.248097 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:00.248639 (+ 542us) server_negotiation.cc:299] Negotiation successful
0620 14:57:00.248799 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260620 14:57:00.249752 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.236551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:44641 (local address 127.7.24.194:46153)
0620 14:57:00.237230 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:00.237282 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:00.238131 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:00.238675 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:00.238682 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:00.239051 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:00.239526 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.239539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.240376 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.240379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.241257 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.241266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.241796 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.241801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.241964 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.242676 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:00.242693 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:00.244420 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:00.246533 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:00.246543 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:00.246565 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:00.246981 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:00.247273 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:00.247278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:00.247283 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:00.247502 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:00.248107 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:00.248117 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:00.248377 (+ 260us) client_negotiation.cc:769] Sending connection context
0620 14:57:00.248533 (+ 156us) client_negotiation.cc:240] Negotiation successful
0620 14:57:00.248806 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":103,"threads_started":1}
I20260620 14:57:00.251081 24652 heartbeater.cc:347] Connected to a master server at 127.7.24.254:44641
I20260620 14:57:00.251358 24652 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:00.252031 24652 heartbeater.cc:511] Master 127.7.24.254:44641 requested a full tablet report, sending...
I20260620 14:57:00.253306 24325 ts_manager.cc:194] Registered new tserver with Master: c97ceb842531433db4af349bdeddc52b (127.7.24.194:44529)
I20260620 14:57:00.253906 24325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:46153
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:57:00.274107 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44641
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:00.385531 24657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:00.385804 24657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:00.385878 24657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:00.389931 24657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:00.390008 24657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:00.390101 24657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:57:00.395288 24657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:44641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:00.396307 24657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:00.397284 24657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:00.406788 24663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:00.406855 24657 server_base.cc:1034] running on GCE node
W20260620 14:57:00.406807 24662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:00.406788 24665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:00.407369 24657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:00.407963 24657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:00.409126 24657 hybrid_clock.cc:648] HybridClock initialized: now 1781967420409089 us; error 52 us; skew 500 ppm
Jun 20 14:57:00 dist-test-slave-sms9 krb5kdc[24282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967420, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:00.411896 24657 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:57:00.412773 24657 webserver.cc:460] Webserver started at http://127.7.24.195:40605/ using document root <none> and password file <none>
I20260620 14:57:00.413326 24657 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:00.413416 24657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:00.413672 24657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:00.415722 24657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "18bd0affe19b47829b6401f3790d3e08"
format_stamp: "Formatted at 2026-06-20 14:57:00 on dist-test-slave-sms9"
server_key: "d7576a2a5f82f64fc5f0183f42e9c482"
server_key_iv: "a4e45fbdde14293d9f5c75640393c797"
server_key_version: "encryptionkey@0"
I20260620 14:57:00.416273 24657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "18bd0affe19b47829b6401f3790d3e08"
format_stamp: "Formatted at 2026-06-20 14:57:00 on dist-test-slave-sms9"
server_key: "d7576a2a5f82f64fc5f0183f42e9c482"
server_key_iv: "a4e45fbdde14293d9f5c75640393c797"
server_key_version: "encryptionkey@0"
I20260620 14:57:00.419937 24657 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:57:00.422339 24672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:00.423777 24657 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:57:00.423919 24657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "18bd0affe19b47829b6401f3790d3e08"
format_stamp: "Formatted at 2026-06-20 14:57:00 on dist-test-slave-sms9"
server_key: "d7576a2a5f82f64fc5f0183f42e9c482"
server_key_iv: "a4e45fbdde14293d9f5c75640393c797"
server_key_version: "encryptionkey@0"
I20260620 14:57:00.424034 24657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:00.445184 24657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:00.448710 24657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:00.448997 24657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:00.449631 24657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:00.450567 24657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:00.450635 24657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:00.450703 24657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:00.450753 24657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:00.459960 24657 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:33267
I20260620 14:57:00.459982 24785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:33267 every 8 connection(s)
I20260620 14:57:00.460886 24657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
Jun 20 14:57:00 dist-test-slave-sms9 krb5kdc[24282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967420, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:00.471010 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 24657
I20260620 14:57:00.471149 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:57:00.471570 7267 external_mini_cluster.cc:1428] Setting key fd7d400075a8dc65efda321568c3eea8
I20260620 14:57:00.475411 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.462758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:37917 (local address 127.7.24.254:44641)
0620 14:57:00.462918 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:00.462923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:00.463783 (+ 860us) server_negotiation.cc:407] Connection header received
0620 14:57:00.465010 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:00.465014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:00.465064 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:00.465198 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:00.466423 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.467037 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.468175 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.468398 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.468508 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.471032 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:00.471036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:00.471039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:00.471072 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:00.472826 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:00.473580 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:00.473583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:00.473584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:00.473690 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:00.474102 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:00.474105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:00.474106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:00.474575 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:00.474688 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:00.475153 (+ 465us) server_negotiation.cc:299] Negotiation successful
0620 14:57:00.475283 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 14:57:00.476157 24788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.463180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:44641 (local address 127.7.24.195:37917)
0620 14:57:00.463622 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:00.463664 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:00.464801 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:00.465424 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:00.465431 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:00.465802 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:00.466270 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.466283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.467170 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.467175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.468036 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.468044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.468545 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.468549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.468718 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.469320 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:00.469336 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:00.470855 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:00.473083 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:00.473089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:00.473103 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:00.473469 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:00.473788 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:00.473790 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:00.473791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:00.473986 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:00.474693 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:00.474699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:00.474929 (+ 230us) client_negotiation.cc:769] Sending connection context
0620 14:57:00.475092 (+ 163us) client_negotiation.cc:240] Negotiation successful
0620 14:57:00.475353 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":106,"threads_started":1}
I20260620 14:57:00.477273 24786 heartbeater.cc:347] Connected to a master server at 127.7.24.254:44641
I20260620 14:57:00.477473 24786 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:00.477948 24786 heartbeater.cc:511] Master 127.7.24.254:44641 requested a full tablet report, sending...
I20260620 14:57:00.478950 24325 ts_manager.cc:194] Registered new tserver with Master: 18bd0affe19b47829b6401f3790d3e08 (127.7.24.195:33267)
I20260620 14:57:00.479519 24325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:37917
I20260620 14:57:00.485699 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:57:00.495231 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.487738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58958 (local address 127.7.24.254:44641)
0620 14:57:00.487887 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:00.487892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:00.488111 (+ 219us) server_negotiation.cc:407] Connection header received
0620 14:57:00.488191 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:00.488193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:00.488232 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:00.488311 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:00.489091 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.489606 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.490516 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.490737 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.490876 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.492211 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:00.492215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:00.492218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:00.492250 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:00.493482 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:00.493962 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:00.493964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:00.493965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:00.494037 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:00.494265 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:00.494267 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:00.494268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:00.494527 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:00.494640 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:00.494913 (+ 273us) server_negotiation.cc:299] Negotiation successful
0620 14:57:00.495017 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 14:57:00.498154 24325 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:57:00.501255 24325 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:57:00.515990 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.510359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:33267 (local address 127.0.0.1:34500)
0620 14:57:00.511174 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:00.511188 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:00.511280 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:00.511947 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:00.511951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:00.511969 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:00.512047 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.512052 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.513392 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.513396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.514509 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.514519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.515552 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.515556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.515641 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.515663 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 14:57:00.515746 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 14:57:00.515819 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":119,"threads_started":1}
I20260620 14:57:00.516023 24802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.510798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34500 (local address 127.7.24.195:33267)
0620 14:57:00.511525 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:00.511536 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:00.511556 (+ 20us) server_negotiation.cc:407] Connection header received
0620 14:57:00.511626 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:00.511631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:00.511794 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:00.511953 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:00.512171 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.513233 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.514618 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.515427 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.515540 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.515664 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:00.515777 (+ 113us) server_negotiation.cc:299] Negotiation successful
0620 14:57:00.515869 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":92,"threads_started":1}
I20260620 14:57:00.516173 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.510432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60626 (local address 127.7.24.194:44529)
0620 14:57:00.510914 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:00.510926 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:00.511265 (+ 339us) server_negotiation.cc:407] Connection header received
0620 14:57:00.511328 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:00.511331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:00.511459 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:00.511605 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:00.511831 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.512681 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.514304 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.515142 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.515274 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.515412 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:00.515905 (+ 493us) server_negotiation.cc:299] Negotiation successful
0620 14:57:00.516005 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":105,"threads_started":1}
I20260620 14:57:00.516196 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.509767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35880 (local address 127.7.24.193:44831)
0620 14:57:00.510305 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:00.510346 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:00.510376 (+ 30us) server_negotiation.cc:407] Connection header received
0620 14:57:00.510484 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:00.510490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:00.510705 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:00.510860 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:00.511415 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.512637 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.514429 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.515085 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.515230 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.515397 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:00.515949 (+ 552us) server_negotiation.cc:299] Negotiation successful
0620 14:57:00.516046 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":107,"threads_started":1}
I20260620 14:57:00.515817 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.509628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:44831 (local address 127.0.0.1:35880)
0620 14:57:00.510098 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:00.510150 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:00.510284 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:00.511068 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:00.511075 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:00.511099 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:00.511272 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.511279 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.512790 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.512793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.514283 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.514294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.515241 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.515244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.515353 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.515390 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 14:57:00.515510 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 14:57:00.515644 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":59,"thread_start_us":85,"threads_started":1}
I20260620 14:57:00.515817 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.509721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:44529 (local address 127.0.0.1:60626)
0620 14:57:00.511051 (+ 1330us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:00.511065 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:00.511164 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:00.511588 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:00.511591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:00.511608 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:00.511707 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.511715 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.512815 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.512819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.514175 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.514185 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.515479 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.515486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.515569 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.515587 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 14:57:00.515652 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 14:57:00.515700 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1267,"mutex_wait_us":51,"thread_start_us":89,"threads_started":1}
I20260620 14:57:00.519337 24586 tablet_service.cc:1467] Processing CreateTablet for tablet 73f3185cae0841f68c425ddef3193bf2 (DEFAULT_TABLE table=test-table [id=354b6631f7d54e9a95573b3d2168eb93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:00.519433 24720 tablet_service.cc:1467] Processing CreateTablet for tablet 73f3185cae0841f68c425ddef3193bf2 (DEFAULT_TABLE table=test-table [id=354b6631f7d54e9a95573b3d2168eb93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:00.520536 24720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f3185cae0841f68c425ddef3193bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:00.520573 24586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f3185cae0841f68c425ddef3193bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:00.520358 24452 tablet_service.cc:1467] Processing CreateTablet for tablet 73f3185cae0841f68c425ddef3193bf2 (DEFAULT_TABLE table=test-table [id=354b6631f7d54e9a95573b3d2168eb93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:00.521283 24452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f3185cae0841f68c425ddef3193bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:00.526432 24803 tablet_bootstrap.cc:492] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: Bootstrap starting.
I20260620 14:57:00.528868 24804 tablet_bootstrap.cc:492] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: Bootstrap starting.
I20260620 14:57:00.529341 24803 tablet_bootstrap.cc:654] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:00.530180 24803 log.cc:824] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:00.530750 24804 tablet_bootstrap.cc:654] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:00.530988 24805 tablet_bootstrap.cc:492] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228: Bootstrap starting.
I20260620 14:57:00.531973 24804 log.cc:824] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:00.532320 24803 tablet_bootstrap.cc:492] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: No bootstrap required, opened a new log
I20260620 14:57:00.532534 24803 ts_tablet_manager.cc:1397] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20260620 14:57:00.533020 24805 tablet_bootstrap.cc:654] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:00.533654 24805 log.cc:824] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:00.535866 24805 tablet_bootstrap.cc:492] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228: No bootstrap required, opened a new log
I20260620 14:57:00.536001 24804 tablet_bootstrap.cc:492] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: No bootstrap required, opened a new log
I20260620 14:57:00.536060 24805 ts_tablet_manager.cc:1397] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 14:57:00.536157 24804 ts_tablet_manager.cc:1397] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260620 14:57:00.536319 24803 raft_consensus.cc:348] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 44831 } } peers { permanent_uuid: "c97ceb842531433db4af349bdeddc52b" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 44529 } } peers { permanent_uuid: "18bd0affe19b47829b6401f3790d3e08" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 33267 } }
I20260620 14:57:00.536578 24803 raft_consensus.cc:374] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:00.536630 24803 raft_consensus.cc:729] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97ceb842531433db4af349bdeddc52b, State: Initialized, Role: FOLLOWER
I20260620 14:57:00.537086 24803 consensus_queue.cc:260] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 44831 } } peers { permanent_uuid: "c97ceb842531433db4af349bdeddc52b" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 44529 } } peers { permanent_uuid: "18bd0affe19b47829b6401f3790d3e08" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 33267 } }
I20260620 14:57:00.538097 24803 ts_tablet_manager.cc:1428] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260620 14:57:00.538504 24652 heartbeater.cc:503] Master 127.7.24.254:44641 was elected leader, sending a full tablet report...
I20260620 14:57:00.540072 24804 raft_consensus.cc:348] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 44831 } } peers { permanent_uuid: "c97ceb842531433db4af349bdeddc52b" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 44529 } } peers { permanent_uuid: "18bd0affe19b47829b6401f3790d3e08" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 33267 } }
I20260620 14:57:00.540361 24804 raft_consensus.cc:374] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:00.540416 24804 raft_consensus.cc:729] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18bd0affe19b47829b6401f3790d3e08, State: Initialized, Role: FOLLOWER
I20260620 14:57:00.540313 24805 raft_consensus.cc:348] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 44831 } } peers { permanent_uuid: "c97ceb842531433db4af349bdeddc52b" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 44529 } } peers { permanent_uuid: "18bd0affe19b47829b6401f3790d3e08" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 33267 } }
I20260620 14:57:00.540546 24805 raft_consensus.cc:374] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:00.540597 24805 raft_consensus.cc:729] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c1b5d7974a482f9dbeedcd726a3228, State: Initialized, Role: FOLLOWER
I20260620 14:57:00.540838 24804 consensus_queue.cc:260] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 44831 } } peers { permanent_uuid: "c97ceb842531433db4af349bdeddc52b" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 44529 } } peers { permanent_uuid: "18bd0affe19b47829b6401f3790d3e08" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 33267 } }
I20260620 14:57:00.540946 24805 consensus_queue.cc:260] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 44831 } } peers { permanent_uuid: "c97ceb842531433db4af349bdeddc52b" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 44529 } } peers { permanent_uuid: "18bd0affe19b47829b6401f3790d3e08" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 33267 } }
I20260620 14:57:00.541481 24786 heartbeater.cc:503] Master 127.7.24.254:44641 was elected leader, sending a full tablet report...
I20260620 14:57:00.541776 24804 ts_tablet_manager.cc:1428] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 14:57:00.541800 24518 heartbeater.cc:503] Master 127.7.24.254:44641 was elected leader, sending a full tablet report...
I20260620 14:57:00.543550 24805 ts_tablet_manager.cc:1428] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260620 14:57:00.711825 24787 tablet.cc:2369] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 14:57:00.735683 24653 tablet.cc:2369] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 14:57:00.764074 24519 tablet.cc:2369] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:57:00.848510 24811 raft_consensus.cc:482] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:57:00.848800 24811 raft_consensus.cc:504] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 44831 } } peers { permanent_uuid: "c97ceb842531433db4af349bdeddc52b" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 44529 } } peers { permanent_uuid: "18bd0affe19b47829b6401f3790d3e08" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 33267 } }
I20260620 14:57:00.850076 24811 leader_election.cc:288] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c97ceb842531433db4af349bdeddc52b (127.7.24.194:44529), 18bd0affe19b47829b6401f3790d3e08 (127.7.24.195:33267)
I20260620 14:57:00.854206 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.850421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:46111 (local address 127.7.24.194:44529)
0620 14:57:00.850686 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:00.850691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:00.850899 (+ 208us) server_negotiation.cc:407] Connection header received
0620 14:57:00.851080 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:00.851083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:00.851137 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:00.851227 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:00.851508 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.852009 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.853181 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.853588 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.853725 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.853779 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:00.853981 (+ 202us) server_negotiation.cc:299] Negotiation successful
0620 14:57:00.854065 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260620 14:57:00.854207 24802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.850549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:46483 (local address 127.7.24.195:33267)
0620 14:57:00.850686 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:00.850691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:00.850960 (+ 269us) server_negotiation.cc:407] Connection header received
0620 14:57:00.851127 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:00.851130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:00.851182 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:00.851307 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:00.851588 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.852081 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.853172 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.853596 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.853740 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.853793 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:00.853986 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 14:57:00.854065 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 14:57:00.854239 24812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.850283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:44529 (local address 127.7.24.193:46111)
0620 14:57:00.850795 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:00.850808 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:00.850961 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:00.851238 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:00.851241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:00.851256 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:00.851390 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.851396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.852124 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.852129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.853047 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.853056 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.853725 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.853729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.853825 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.853855 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 14:57:00.853961 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 14:57:00.854085 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":83,"thread_start_us":119,"threads_started":1}
I20260620 14:57:00.854243 24813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.850402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:33267 (local address 127.7.24.193:46483)
0620 14:57:00.850851 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:00.850864 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:00.851001 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:00.851320 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:00.851322 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:00.851338 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:00.851472 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.851478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.852197 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.852202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.853041 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:00.853051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.853734 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.853738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:00.853861 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.853878 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 14:57:00.853968 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 14:57:00.854076 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":6,"thread_start_us":132,"threads_started":1}
I20260620 14:57:00.855010 24606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f3185cae0841f68c425ddef3193bf2" candidate_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97ceb842531433db4af349bdeddc52b" is_pre_election: true
I20260620 14:57:00.855134 24740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f3185cae0841f68c425ddef3193bf2" candidate_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18bd0affe19b47829b6401f3790d3e08" is_pre_election: true
I20260620 14:57:00.855361 24606 raft_consensus.cc:2454] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c1b5d7974a482f9dbeedcd726a3228 in term 0.
I20260620 14:57:00.855458 24740 raft_consensus.cc:2454] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c1b5d7974a482f9dbeedcd726a3228 in term 0.
I20260620 14:57:00.855921 24405 leader_election.cc:302] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8c1b5d7974a482f9dbeedcd726a3228, c97ceb842531433db4af349bdeddc52b; no voters:
I20260620 14:57:00.856217 24811 raft_consensus.cc:2781] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:57:00.856307 24811 raft_consensus.cc:482] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:57:00.856351 24811 raft_consensus.cc:3037] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:00.857409 24811 raft_consensus.cc:504] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 44831 } } peers { permanent_uuid: "c97ceb842531433db4af349bdeddc52b" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 44529 } } peers { permanent_uuid: "18bd0affe19b47829b6401f3790d3e08" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 33267 } }
I20260620 14:57:00.857787 24811 leader_election.cc:288] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [CANDIDATE]: Term 1 election: Requested vote from peers c97ceb842531433db4af349bdeddc52b (127.7.24.194:44529), 18bd0affe19b47829b6401f3790d3e08 (127.7.24.195:33267)
I20260620 14:57:00.858202 24606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f3185cae0841f68c425ddef3193bf2" candidate_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c97ceb842531433db4af349bdeddc52b"
I20260620 14:57:00.858258 24740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f3185cae0841f68c425ddef3193bf2" candidate_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18bd0affe19b47829b6401f3790d3e08"
I20260620 14:57:00.858389 24606 raft_consensus.cc:3037] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:00.858387 24740 raft_consensus.cc:3037] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:00.859477 24740 raft_consensus.cc:2454] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c1b5d7974a482f9dbeedcd726a3228 in term 1.
I20260620 14:57:00.859495 24606 raft_consensus.cc:2454] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c1b5d7974a482f9dbeedcd726a3228 in term 1.
I20260620 14:57:00.859803 24407 leader_election.cc:302] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18bd0affe19b47829b6401f3790d3e08, b8c1b5d7974a482f9dbeedcd726a3228; no voters:
I20260620 14:57:00.860018 24811 raft_consensus.cc:2781] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:00.860256 24811 raft_consensus.cc:686] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 1 LEADER]: Becoming Leader. State: Replica: b8c1b5d7974a482f9dbeedcd726a3228, State: Running, Role: LEADER
I20260620 14:57:00.860606 24811 consensus_queue.cc:237] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 44831 } } peers { permanent_uuid: "c97ceb842531433db4af349bdeddc52b" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 44529 } } peers { permanent_uuid: "18bd0affe19b47829b6401f3790d3e08" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 33267 } }
I20260620 14:57:00.864053 24325 catalog_manager.cc:5495] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c1b5d7974a482f9dbeedcd726a3228 (127.7.24.193). New cstate: current_term: 1 leader_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8c1b5d7974a482f9dbeedcd726a3228" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 44831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c97ceb842531433db4af349bdeddc52b" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 44529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18bd0affe19b47829b6401f3790d3e08" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 33267 } health_report { overall_health: UNKNOWN } } }
I20260620 14:57:00.885411 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.882013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.7.24.193:44831)
0620 14:57:00.882146 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:00.882150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:00.882163 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:57:00.882279 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:00.882282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:00.882338 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:00.882423 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:00.882722 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.883230 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.884087 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:00.884293 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:00.884370 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:00.884624 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:00.885050 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:00.885141 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:00.885245 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 14:57:00.885296 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 14:57:00.891464 24606 raft_consensus.cc:1261] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b [term 1 FOLLOWER]: Refusing update from remote peer b8c1b5d7974a482f9dbeedcd726a3228: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:00.891459 24740 raft_consensus.cc:1261] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08 [term 1 FOLLOWER]: Refusing update from remote peer b8c1b5d7974a482f9dbeedcd726a3228: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:00.892211 24811 consensus_queue.cc:1035] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c97ceb842531433db4af349bdeddc52b" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 44529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:00.892412 24814 consensus_queue.cc:1035] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bd0affe19b47829b6401f3790d3e08" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 33267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:00.899776 24817 mvcc.cc:205] Tried to move back new op lower bound from 7298938555963838464 to 7298938555847786496. Current Snapshot: MvccSnapshot[applied={T|T < 7298938555963838464}]
I20260620 14:57:00.899995 24819 mvcc.cc:205] Tried to move back new op lower bound from 7298938555963838464 to 7298938555847786496. Current Snapshot: MvccSnapshot[applied={T|T < 7298938555963838464}]
I20260620 14:57:00.900429 24818 mvcc.cc:205] Tried to move back new op lower bound from 7298938555963838464 to 7298938555847786496. Current Snapshot: MvccSnapshot[applied={T|T < 7298938555963838464}]
I20260620 14:57:00.911263 24327 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:00.911427 24327 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:00.913986 24327 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df43ad3d2d434953897c8fa082f963af: Sending DeleteTablet for 3 replicas of tablet 73f3185cae0841f68c425ddef3193bf2
I20260620 14:57:00.914871 24586 tablet_service.cc:1514] Processing DeleteTablet for tablet 73f3185cae0841f68c425ddef3193bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:00 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:60626
I20260620 14:57:00.914872 24452 tablet_service.cc:1514] Processing DeleteTablet for tablet 73f3185cae0841f68c425ddef3193bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:00 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:35880
I20260620 14:57:00.915184 24720 tablet_service.cc:1514] Processing DeleteTablet for tablet 73f3185cae0841f68c425ddef3193bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:00 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:34500
I20260620 14:57:00.915585 24827 tablet_replica.cc:330] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228: stopping tablet replica
I20260620 14:57:00.915620 24828 tablet_replica.cc:330] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: stopping tablet replica
I20260620 14:57:00.915902 24826 tablet_replica.cc:330] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: stopping tablet replica
I20260620 14:57:00.915910 24827 raft_consensus.cc:2229] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:57:00.915910 24828 raft_consensus.cc:2229] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:57:00.916242 24828 raft_consensus.cc:2258] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:00.916270 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 24389
I20260620 14:57:00.916244 24826 raft_consensus.cc:2229] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:57:00.916412 24827 raft_consensus.cc:2258] T 73f3185cae0841f68c425ddef3193bf2 P b8c1b5d7974a482f9dbeedcd726a3228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:00.916676 24826 raft_consensus.cc:2258] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:00.918543 24826 ts_tablet_manager.cc:1905] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:00.918543 24828 ts_tablet_manager.cc:1905] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:00.921242 24828 ts_tablet_manager.cc:1918] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:57:00.921242 24826 ts_tablet_manager.cc:1918] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:57:00.921353 24826 log.cc:1196] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/73f3185cae0841f68c425ddef3193bf2
I20260620 14:57:00.921353 24828 log.cc:1196] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/73f3185cae0841f68c425ddef3193bf2
I20260620 14:57:00.921690 24828 ts_tablet_manager.cc:1939] T 73f3185cae0841f68c425ddef3193bf2 P 18bd0affe19b47829b6401f3790d3e08: Deleting consensus metadata
I20260620 14:57:00.921690 24826 ts_tablet_manager.cc:1939] T 73f3185cae0841f68c425ddef3193bf2 P c97ceb842531433db4af349bdeddc52b: Deleting consensus metadata
I20260620 14:57:00.923101 24313 catalog_manager.cc:4841] TS 18bd0affe19b47829b6401f3790d3e08 (127.7.24.195:33267): tablet 73f3185cae0841f68c425ddef3193bf2 (table test-table [id=354b6631f7d54e9a95573b3d2168eb93]) successfully deleted
I20260620 14:57:00.923135 24311 catalog_manager.cc:4841] TS c97ceb842531433db4af349bdeddc52b (127.7.24.194:44529): tablet 73f3185cae0841f68c425ddef3193bf2 (table test-table [id=354b6631f7d54e9a95573b3d2168eb93]) successfully deleted
W20260620 14:57:00.924145 24313 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:44831 (error 108)
I20260620 14:57:00.924394 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 24523
I20260620 14:57:00.925483 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:00.925191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:44831 (local address 127.0.0.1:35890)
0620 14:57:00.925328 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:00.925405 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:44831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260620 14:57:00.925671 24313 catalog_manager.cc:4584] TS b8c1b5d7974a482f9dbeedcd726a3228 (127.7.24.193:44831): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73f3185cae0841f68c425ddef3193bf2: Network error: Client connection negotiation failed: client connection to 127.7.24.193:44831: connect: Connection refused (error 111)
I20260620 14:57:00.931252 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 24657
I20260620 14:57:00.938411 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 24295
2026-06-20T14:57:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2548 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-20T14:57:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:00Z Disabled control of system clock
I20260620 14:57:00.969302 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:34769
--webserver_interface=127.7.24.254
--webserver_port=0
--builtin_ntp_servers=127.7.24.212:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:34769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 14:57:01.079406 24833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:01.079704 24833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:01.079741 24833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:01.083796 24833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:01.083869 24833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:01.083885 24833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:01.083902 24833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:01.083918 24833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:01.088716 24833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:34769
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:34769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:01.089782 24833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:01.091212 24833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:01.099879 24839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:01.099879 24841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:01.099912 24838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:01.099910 24833 server_base.cc:1034] running on GCE node
I20260620 14:57:01.100725 24833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:01.101892 24833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:01.103062 24833 hybrid_clock.cc:648] HybridClock initialized: now 1781967421103034 us; error 41 us; skew 500 ppm
I20260620 14:57:01.105163 24833 webserver.cc:460] Webserver started at http://127.7.24.254:41439/ using document root <none> and password file <none>
I20260620 14:57:01.105753 24833 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:01.105816 24833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:01.106061 24833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:01.108214 24833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "3b918607b0ac4f2892d20d52fe4e1f74"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "04575f5d4ecf6794c06f67f4dce8c035"
server_key_iv: "10e7a66d10212a6186ec5338d11dded4"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.108803 24833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "3b918607b0ac4f2892d20d52fe4e1f74"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "04575f5d4ecf6794c06f67f4dce8c035"
server_key_iv: "10e7a66d10212a6186ec5338d11dded4"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.112612 24833 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260620 14:57:01.115224 24847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:01.116429 24833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 14:57:01.116596 24833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "3b918607b0ac4f2892d20d52fe4e1f74"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "04575f5d4ecf6794c06f67f4dce8c035"
server_key_iv: "10e7a66d10212a6186ec5338d11dded4"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.116721 24833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:01.150108 24833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:01.150929 24833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:01.151147 24833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:01.159346 24833 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:34769
I20260620 14:57:01.159380 24899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:34769 every 8 connection(s)
I20260620 14:57:01.160476 24833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:01.164031 24900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:01.165750 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 24833
I20260620 14:57:01.165915 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:01.166227 7267 external_mini_cluster.cc:1428] Setting key 2e7d757764e54dbeea454ddef6c2ea1f
I20260620 14:57:01.171013 24900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74: Bootstrap starting.
I20260620 14:57:01.174046 24900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:01.174991 24900 log.cc:824] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:01.176069 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.167762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33402 (local address 127.7.24.254:34769)
0620 14:57:01.168392 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:01.168407 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:01.168452 (+ 45us) server_negotiation.cc:407] Connection header received
0620 14:57:01.169115 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:01.169132 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:01.169532 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:01.169810 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:01.170194 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.171276 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.172510 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.173045 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.173172 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.174077 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:01.174082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:01.174096 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:01.174632 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:01.174691 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:01.174713 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:01.174795 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:01.174978 (+ 183us) server_negotiation.cc:299] Negotiation successful
0620 14:57:01.175288 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":140,"threads_started":1}
I20260620 14:57:01.177352 24900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74: No bootstrap required, opened a new log
I20260620 14:57:01.181119 24900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b918607b0ac4f2892d20d52fe4e1f74" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34769 } }
I20260620 14:57:01.181393 24900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:01.181483 24900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b918607b0ac4f2892d20d52fe4e1f74, State: Initialized, Role: FOLLOWER
I20260620 14:57:01.181910 24900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b918607b0ac4f2892d20d52fe4e1f74" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34769 } }
I20260620 14:57:01.182063 24900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:01.182149 24900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:01.182277 24900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:01.183331 24900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b918607b0ac4f2892d20d52fe4e1f74" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34769 } }
I20260620 14:57:01.183683 24900 leader_election.cc:302] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b918607b0ac4f2892d20d52fe4e1f74; no voters:
I20260620 14:57:01.184067 24900 leader_election.cc:288] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:01.184185 24905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:01.184396 24905 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [term 1 LEADER]: Becoming Leader. State: Replica: 3b918607b0ac4f2892d20d52fe4e1f74, State: Running, Role: LEADER
I20260620 14:57:01.184734 24905 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b918607b0ac4f2892d20d52fe4e1f74" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34769 } }
I20260620 14:57:01.185372 24900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:01.186553 24907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b918607b0ac4f2892d20d52fe4e1f74. Latest consensus state: current_term: 1 leader_uuid: "3b918607b0ac4f2892d20d52fe4e1f74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b918607b0ac4f2892d20d52fe4e1f74" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34769 } } }
I20260620 14:57:01.186669 24907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:01.186360 24906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b918607b0ac4f2892d20d52fe4e1f74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b918607b0ac4f2892d20d52fe4e1f74" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 34769 } } }
I20260620 14:57:01.186897 24906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:01.187141 24914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:01.189855 24914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:01.195873 24914 catalog_manager.cc:1347] Generated new cluster ID: af8b49a5f0bc41dd8530eacd1f42b578
I20260620 14:57:01.195984 24914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:01.208746 24914 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:01.209703 24914 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:01.219247 24914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74: Generated new TSK 0
I20260620 14:57:01.220225 24914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:01.231102 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34769
--builtin_ntp_servers=127.7.24.212:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 14:57:01.346680 24924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:01.346953 24924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:01.347021 24924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:01.351321 24924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:01.351377 24924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:01.351464 24924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:57:01.356396 24924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:01.357506 24924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:01.358950 24924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:01.368742 24929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:01.368851 24924 server_base.cc:1034] running on GCE node
W20260620 14:57:01.368774 24932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:01.368742 24930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:01.369462 24924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:01.370128 24924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:01.371310 24924 hybrid_clock.cc:648] HybridClock initialized: now 1781967421371275 us; error 47 us; skew 500 ppm
I20260620 14:57:01.373133 24924 webserver.cc:460] Webserver started at http://127.7.24.193:34355/ using document root <none> and password file <none>
I20260620 14:57:01.373730 24924 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:01.373826 24924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:01.374064 24924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:01.376245 24924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "6b5533003c01437dba728dd4180e8150"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "16a07b988d32d31b786c063abe24e2eb"
server_key_iv: "e6b46264d92cf57f1d9eefac6d7488e3"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.376835 24924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "6b5533003c01437dba728dd4180e8150"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "16a07b988d32d31b786c063abe24e2eb"
server_key_iv: "e6b46264d92cf57f1d9eefac6d7488e3"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.380570 24924 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260620 14:57:01.383077 24938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:01.384279 24924 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 14:57:01.384449 24924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "6b5533003c01437dba728dd4180e8150"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "16a07b988d32d31b786c063abe24e2eb"
server_key_iv: "e6b46264d92cf57f1d9eefac6d7488e3"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.384573 24924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:01.409904 24924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:01.410691 24924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:01.410907 24924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:01.411530 24924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:01.412509 24924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:01.412578 24924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:01.412652 24924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:01.412700 24924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:01.422595 24924 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:43955
I20260620 14:57:01.422605 25051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:43955 every 8 connection(s)
I20260620 14:57:01.423638 24924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:57:01.428054 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 24924
I20260620 14:57:01.428205 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:57:01.428555 7267 external_mini_cluster.cc:1428] Setting key 3c8a51b2a718f93152462c10940ec8c1
I20260620 14:57:01.430732 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34769
--builtin_ntp_servers=127.7.24.212:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 14:57:01.435097 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.425741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:53465 (local address 127.7.24.254:34769)
0620 14:57:01.425893 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:01.425897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:01.427099 (+ 1202us) server_negotiation.cc:407] Connection header received
0620 14:57:01.428973 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:01.428976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:01.429042 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:01.429157 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:01.430319 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.431176 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.432818 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.433053 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.433138 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.434396 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:01.434402 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:01.434409 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:01.434429 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:01.434455 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:01.434501 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:01.434584 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:01.434776 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 14:57:01.434864 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 14:57:01.436002 25054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.426167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:34769 (local address 127.7.24.193:53465)
0620 14:57:01.426941 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:01.426986 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:01.428733 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:01.429343 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:01.429350 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:01.429442 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:01.430147 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.430165 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.431375 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.431378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.432652 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.432666 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.433235 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.433239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.433405 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.434106 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:57:01.434107 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:01.434149 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:57:01.434221 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:01.434591 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:01.434601 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:01.434606 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 14:57:01.434738 (+ 132us) client_negotiation.cc:240] Negotiation successful
0620 14:57:01.435000 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":117,"threads_started":1}
I20260620 14:57:01.437415 25052 heartbeater.cc:347] Connected to a master server at 127.7.24.254:34769
I20260620 14:57:01.437716 25052 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:01.438264 25052 heartbeater.cc:511] Master 127.7.24.254:34769 requested a full tablet report, sending...
I20260620 14:57:01.440200 24864 ts_manager.cc:194] Registered new tserver with Master: 6b5533003c01437dba728dd4180e8150 (127.7.24.193:43955)
I20260620 14:57:01.442078 24864 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.193:53465
W20260620 14:57:01.552819 25055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:01.553074 25055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:01.553114 25055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:01.557049 25055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:01.557104 25055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:01.557158 25055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:57:01.561971 25055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:01.563052 25055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:01.564495 25055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:57:01.574604 25055 server_base.cc:1034] running on GCE node
W20260620 14:57:01.574548 25060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:01.574579 25063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:01.574542 25061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:01.575273 25055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:01.575945 25055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:01.577144 25055 hybrid_clock.cc:648] HybridClock initialized: now 1781967421577110 us; error 47 us; skew 500 ppm
I20260620 14:57:01.579288 25055 webserver.cc:460] Webserver started at http://127.7.24.194:37307/ using document root <none> and password file <none>
I20260620 14:57:01.579963 25055 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:01.580070 25055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:01.580336 25055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:01.582500 25055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "498f4760e0b343cea254a5dbfe5ccf3a"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "3816bbffb95de6889370463835c5e8c7"
server_key_iv: "521cce3dd1c08c1f53070d22624f3192"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.583105 25055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "498f4760e0b343cea254a5dbfe5ccf3a"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "3816bbffb95de6889370463835c5e8c7"
server_key_iv: "521cce3dd1c08c1f53070d22624f3192"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.587510 25055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 14:57:01.590183 25069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:01.591586 25055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 14:57:01.591734 25055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "498f4760e0b343cea254a5dbfe5ccf3a"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "3816bbffb95de6889370463835c5e8c7"
server_key_iv: "521cce3dd1c08c1f53070d22624f3192"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.591862 25055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:01.606643 25055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:01.607347 25055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:01.607514 25055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:01.608038 25055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:01.608959 25055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:01.609005 25055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:01.609046 25055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:01.609062 25055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:01.618274 25055 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:39097
I20260620 14:57:01.618295 25182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:39097 every 8 connection(s)
I20260620 14:57:01.619302 25055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:57:01.628543 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.621182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:47903 (local address 127.7.24.254:34769)
0620 14:57:01.621315 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:01.621319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:01.622389 (+ 1070us) server_negotiation.cc:407] Connection header received
0620 14:57:01.623798 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:01.623801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:01.623850 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:01.623929 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:01.624817 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.625357 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.626612 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.626827 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.626921 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.628030 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:01.628033 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:01.628035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:01.628047 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:01.628063 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:01.628070 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:01.628128 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:01.628350 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 14:57:01.628416 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 14:57:01.629009 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 25055
I20260620 14:57:01.629148 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:57:01.629501 7267 external_mini_cluster.cc:1428] Setting key 123c91d59377cca2b95a6c121fefc2ed
I20260620 14:57:01.630076 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.621607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:34769 (local address 127.7.24.194:47903)
0620 14:57:01.622231 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:01.622270 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:01.623606 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:01.624087 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:01.624094 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:01.624175 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:01.624651 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.624663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.625494 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.625497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.626393 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.626403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.626972 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.626974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.627121 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.627780 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:57:01.627781 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:01.627822 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:57:01.627893 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:01.628143 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:01.628149 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:01.628153 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 14:57:01.628248 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 14:57:01.628487 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":99,"threads_started":1}
I20260620 14:57:01.631275 25183 heartbeater.cc:347] Connected to a master server at 127.7.24.254:34769
I20260620 14:57:01.631533 25183 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:01.631644 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34769
--builtin_ntp_servers=127.7.24.212:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 14:57:01.632174 25183 heartbeater.cc:511] Master 127.7.24.254:34769 requested a full tablet report, sending...
I20260620 14:57:01.633253 24863 ts_manager.cc:194] Registered new tserver with Master: 498f4760e0b343cea254a5dbfe5ccf3a (127.7.24.194:39097)
I20260620 14:57:01.633862 24863 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.194:47903
W20260620 14:57:01.745476 25186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:01.745810 25186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:01.745894 25186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:01.750185 25186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:01.750293 25186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:01.750391 25186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:57:01.755436 25186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:01.756639 25186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:01.758113 25186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:01.768110 25192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:01.768110 25191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:01.768172 25194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:01.768435 25186 server_base.cc:1034] running on GCE node
I20260620 14:57:01.769029 25186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:01.769724 25186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:01.770920 25186 hybrid_clock.cc:648] HybridClock initialized: now 1781967421770886 us; error 48 us; skew 500 ppm
I20260620 14:57:01.777113 25186 webserver.cc:460] Webserver started at http://127.7.24.195:41353/ using document root <none> and password file <none>
I20260620 14:57:01.777931 25186 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:01.778034 25186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:01.778290 25186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:01.780979 25186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "c8ee197997ca47929b90a868fa26ea98"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "841abfb27992b3666d038b69130590ed"
server_key_iv: "ea5fe67d0c6ac0db63f8568643f7b415"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.781778 25186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "c8ee197997ca47929b90a868fa26ea98"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "841abfb27992b3666d038b69130590ed"
server_key_iv: "ea5fe67d0c6ac0db63f8568643f7b415"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.788003 25186 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260620 14:57:01.792068 25200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:01.794071 25186 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260620 14:57:01.794234 25186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "c8ee197997ca47929b90a868fa26ea98"
format_stamp: "Formatted at 2026-06-20 14:57:01 on dist-test-slave-sms9"
server_key: "841abfb27992b3666d038b69130590ed"
server_key_iv: "ea5fe67d0c6ac0db63f8568643f7b415"
server_key_version: "encryptionkey@0"
I20260620 14:57:01.794400 25186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:01.820298 25186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:01.821300 25186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:01.821522 25186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:01.822350 25186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:01.823808 25186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:01.823876 25186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:01.823949 25186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:01.823989 25186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:01.835021 25186 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:45045
I20260620 14:57:01.835039 25313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:45045 every 8 connection(s)
I20260620 14:57:01.836022 25186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:57:01.838820 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 25186
I20260620 14:57:01.838941 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:57:01.839229 7267 external_mini_cluster.cc:1428] Setting key ae30959853b8994c4729a143392fbac7
I20260620 14:57:01.846751 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.838089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:38095 (local address 127.7.24.254:34769)
0620 14:57:01.838288 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:01.838292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:01.839638 (+ 1346us) server_negotiation.cc:407] Connection header received
0620 14:57:01.841411 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:01.841414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:01.841470 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:01.841546 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:01.842500 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.843056 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.844508 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.844759 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.844898 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.846139 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:01.846143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:01.846146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:01.846159 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:01.846189 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:01.846199 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:01.846275 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:01.846491 (+ 216us) server_negotiation.cc:299] Negotiation successful
0620 14:57:01.846559 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260620 14:57:01.847723 25316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.838673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:34769 (local address 127.7.24.195:38095)
0620 14:57:01.839473 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:01.839527 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:01.841181 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:01.841720 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:01.841726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:01.841815 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:01.842321 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.842333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.843208 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.843212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.844351 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.844363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.844947 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.844951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.845128 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.845851 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:57:01.845852 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:01.845897 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:57:01.845969 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:01.846309 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:01.846317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:01.846322 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 14:57:01.846451 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 14:57:01.846797 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":148,"threads_started":1}
I20260620 14:57:01.849009 25314 heartbeater.cc:347] Connected to a master server at 127.7.24.254:34769
I20260620 14:57:01.849294 25314 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:01.849901 25314 heartbeater.cc:511] Master 127.7.24.254:34769 requested a full tablet report, sending...
I20260620 14:57:01.851156 24863 ts_manager.cc:194] Registered new tserver with Master: c8ee197997ca47929b90a868fa26ea98 (127.7.24.195:45045)
I20260620 14:57:01.851807 24863 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.195:38095
I20260620 14:57:01.853796 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:57:01.859973 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.856125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.7.24.254:34769)
0620 14:57:01.856273 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:01.856279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:01.856469 (+ 190us) server_negotiation.cc:407] Connection header received
0620 14:57:01.856589 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:01.856591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:01.856633 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:01.856702 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:01.856932 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.857437 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.858253 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.858606 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.858691 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.859502 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:01.859505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:01.859508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:01.859520 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:01.859539 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:01.859547 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:01.859609 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:01.859771 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 14:57:01.859829 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 14:57:01.863444 24863 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:57:01.866621 24863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:57:01.882220 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.876043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41888 (local address 127.7.24.194:39097)
0620 14:57:01.876492 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:01.876504 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:01.877098 (+ 594us) server_negotiation.cc:407] Connection header received
0620 14:57:01.877189 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:01.877195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:01.877441 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:01.877603 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:01.877921 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.879244 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.880847 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.881489 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.881617 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.881843 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:01.881943 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 14:57:01.882042 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":117,"threads_started":1}
I20260620 14:57:01.882215 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.875925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:39097 (local address 127.0.0.1:41888)
0620 14:57:01.876762 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:01.876820 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:01.876951 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:01.877591 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:01.877598 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:01.877620 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:01.877792 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.877799 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.879394 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.879398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.880711 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.880722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.881619 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.881623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.881742 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.881783 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 14:57:01.881914 (+ 131us) client_negotiation.cc:240] Negotiation successful
0620 14:57:01.882042 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":90,"thread_start_us":94,"threads_started":1}
I20260620 14:57:01.882539 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.876314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:43955 (local address 127.0.0.1:42328)
0620 14:57:01.877396 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:01.877411 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:01.877554 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:01.877996 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:01.878001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:01.878020 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:01.878110 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.878115 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.879505 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.879508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.881025 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.881038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.882127 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.882130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.882217 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.882240 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 14:57:01.882291 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 14:57:01.882376 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"mutex_wait_us":227,"thread_start_us":87,"threads_started":1}
I20260620 14:57:01.882959 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.875979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:45045 (local address 127.0.0.1:52232)
0620 14:57:01.877252 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:01.877265 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:01.877342 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:01.878197 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:01.878200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:01.878217 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:01.878305 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.878312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.879330 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.879336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.881039 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.881046 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.882337 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.882340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.882604 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.882635 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 14:57:01.882713 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 14:57:01.882784 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1200,"mutex_wait_us":119,"thread_start_us":130,"threads_started":1}
I20260620 14:57:01.883399 25330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.876744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.7.24.195:45045)
0620 14:57:01.877810 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:01.877821 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:01.877842 (+ 21us) server_negotiation.cc:407] Connection header received
0620 14:57:01.877914 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:01.877922 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:01.878079 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:01.878214 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:01.878424 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.879211 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.881175 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.881986 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.882103 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.882278 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:01.883138 (+ 860us) server_negotiation.cc:299] Negotiation successful
0620 14:57:01.883243 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":171,"threads_started":1}
I20260620 14:57:01.883445 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.876739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.7.24.193:43955)
0620 14:57:01.877234 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:01.877248 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:01.877526 (+ 278us) server_negotiation.cc:407] Connection header received
0620 14:57:01.877665 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:01.877671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:01.877860 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:01.878004 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:01.878247 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.879387 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.881175 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.881986 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.882864 (+ 878us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.883171 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:01.883216 (+ 45us) server_negotiation.cc:299] Negotiation successful
0620 14:57:01.883299 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":125,"threads_started":1}
I20260620 14:57:01.885655 25248 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa9609774dd49fb87dcbcac217d528f (DEFAULT_TABLE table=test-table [id=01ff9172ec674ece93f50f176a90b719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:01.885695 25117 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa9609774dd49fb87dcbcac217d528f (DEFAULT_TABLE table=test-table [id=01ff9172ec674ece93f50f176a90b719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:01.886703 25248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa9609774dd49fb87dcbcac217d528f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:01.886729 25117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa9609774dd49fb87dcbcac217d528f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:01.886538 24986 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa9609774dd49fb87dcbcac217d528f (DEFAULT_TABLE table=test-table [id=01ff9172ec674ece93f50f176a90b719]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:01.887468 24986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa9609774dd49fb87dcbcac217d528f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:01.892969 25331 tablet_bootstrap.cc:492] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98: Bootstrap starting.
I20260620 14:57:01.894805 25332 tablet_bootstrap.cc:492] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a: Bootstrap starting.
I20260620 14:57:01.895085 25333 tablet_bootstrap.cc:492] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150: Bootstrap starting.
I20260620 14:57:01.895830 25331 tablet_bootstrap.cc:654] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:01.896764 25331 log.cc:824] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:01.897327 25332 tablet_bootstrap.cc:654] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:01.897656 25333 tablet_bootstrap.cc:654] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:01.898070 25332 log.cc:824] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:01.898381 25333 log.cc:824] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:01.899029 25331 tablet_bootstrap.cc:492] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98: No bootstrap required, opened a new log
I20260620 14:57:01.899245 25331 ts_tablet_manager.cc:1397] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 14:57:01.900143 25332 tablet_bootstrap.cc:492] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a: No bootstrap required, opened a new log
I20260620 14:57:01.900166 25333 tablet_bootstrap.cc:492] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150: No bootstrap required, opened a new log
I20260620 14:57:01.900321 25332 ts_tablet_manager.cc:1397] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:57:01.900364 25333 ts_tablet_manager.cc:1397] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260620 14:57:01.903924 25331 raft_consensus.cc:348] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39097 } } peers { permanent_uuid: "c8ee197997ca47929b90a868fa26ea98" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45045 } } peers { permanent_uuid: "6b5533003c01437dba728dd4180e8150" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43955 } }
I20260620 14:57:01.904006 25333 raft_consensus.cc:348] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39097 } } peers { permanent_uuid: "c8ee197997ca47929b90a868fa26ea98" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45045 } } peers { permanent_uuid: "6b5533003c01437dba728dd4180e8150" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43955 } }
I20260620 14:57:01.904223 25331 raft_consensus.cc:374] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:01.904237 25333 raft_consensus.cc:374] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:01.904294 25331 raft_consensus.cc:729] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8ee197997ca47929b90a868fa26ea98, State: Initialized, Role: FOLLOWER
I20260620 14:57:01.904304 25333 raft_consensus.cc:729] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b5533003c01437dba728dd4180e8150, State: Initialized, Role: FOLLOWER
I20260620 14:57:01.904503 25332 raft_consensus.cc:348] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39097 } } peers { permanent_uuid: "c8ee197997ca47929b90a868fa26ea98" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45045 } } peers { permanent_uuid: "6b5533003c01437dba728dd4180e8150" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43955 } }
I20260620 14:57:01.904733 25332 raft_consensus.cc:374] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:01.904793 25332 raft_consensus.cc:729] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498f4760e0b343cea254a5dbfe5ccf3a, State: Initialized, Role: FOLLOWER
I20260620 14:57:01.904798 25331 consensus_queue.cc:260] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39097 } } peers { permanent_uuid: "c8ee197997ca47929b90a868fa26ea98" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45045 } } peers { permanent_uuid: "6b5533003c01437dba728dd4180e8150" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43955 } }
I20260620 14:57:01.904803 25333 consensus_queue.cc:260] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39097 } } peers { permanent_uuid: "c8ee197997ca47929b90a868fa26ea98" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45045 } } peers { permanent_uuid: "6b5533003c01437dba728dd4180e8150" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43955 } }
I20260620 14:57:01.905217 25332 consensus_queue.cc:260] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39097 } } peers { permanent_uuid: "c8ee197997ca47929b90a868fa26ea98" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45045 } } peers { permanent_uuid: "6b5533003c01437dba728dd4180e8150" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43955 } }
I20260620 14:57:01.905727 25052 heartbeater.cc:503] Master 127.7.24.254:34769 was elected leader, sending a full tablet report...
I20260620 14:57:01.905921 25333 ts_tablet_manager.cc:1428] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260620 14:57:01.906035 25332 ts_tablet_manager.cc:1428] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 14:57:01.906641 25183 heartbeater.cc:503] Master 127.7.24.254:34769 was elected leader, sending a full tablet report...
I20260620 14:57:01.906884 25314 heartbeater.cc:503] Master 127.7.24.254:34769 was elected leader, sending a full tablet report...
I20260620 14:57:01.907815 25331 ts_tablet_manager.cc:1428] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
W20260620 14:57:01.924971 25053 tablet.cc:2369] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:57:01.960651 25338 raft_consensus.cc:482] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:57:01.960937 25338 raft_consensus.cc:504] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39097 } } peers { permanent_uuid: "c8ee197997ca47929b90a868fa26ea98" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45045 } } peers { permanent_uuid: "6b5533003c01437dba728dd4180e8150" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43955 } }
I20260620 14:57:01.962148 25338 leader_election.cc:288] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 498f4760e0b343cea254a5dbfe5ccf3a (127.7.24.194:39097), c8ee197997ca47929b90a868fa26ea98 (127.7.24.195:45045)
I20260620 14:57:01.966562 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.962903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:59763 (local address 127.7.24.194:39097)
0620 14:57:01.963038 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:01.963044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:01.963062 (+ 18us) server_negotiation.cc:407] Connection header received
0620 14:57:01.963204 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:01.963206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:01.963272 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:01.963350 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:01.963668 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.964242 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.965388 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.965904 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.966039 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.966090 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:01.966310 (+ 220us) server_negotiation.cc:299] Negotiation successful
0620 14:57:01.966391 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 14:57:01.966583 25330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.962714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:48565 (local address 127.7.24.195:45045)
0620 14:57:01.962869 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:01.962875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:01.962965 (+ 90us) server_negotiation.cc:407] Connection header received
0620 14:57:01.963130 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:01.963135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:01.963197 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:01.963263 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:01.963789 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.964473 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.965533 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.965933 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.966069 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.966121 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:01.966321 (+ 200us) server_negotiation.cc:299] Negotiation successful
0620 14:57:01.966400 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 14:57:01.966604 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.962500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:45045 (local address 127.7.24.193:48565)
0620 14:57:01.962847 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:01.962865 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:01.963007 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:01.963359 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:01.963362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:01.963378 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:01.963633 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.963646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.964587 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.964593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.965406 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.965415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.966069 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.966072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.966196 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.966213 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 14:57:01.966300 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 14:57:01.966400 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":28,"spinlock_wait_cycles":23424,"thread_start_us":105,"threads_started":1}
I20260620 14:57:01.966606 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:01.962500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:39097 (local address 127.7.24.193:59763)
0620 14:57:01.962957 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:01.962976 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:01.963089 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:01.963387 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:01.963390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:01.963433 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:01.963530 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.963535 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.964370 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.964374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.965254 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:01.965263 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:01.966033 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:01.966037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:01.966135 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:01.966203 (+ 68us) client_negotiation.cc:769] Sending connection context
0620 14:57:01.966295 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 14:57:01.966413 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":111,"spinlock_wait_cycles":768,"thread_start_us":107,"threads_started":1}
I20260620 14:57:01.967329 25268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa9609774dd49fb87dcbcac217d528f" candidate_uuid: "6b5533003c01437dba728dd4180e8150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ee197997ca47929b90a868fa26ea98" is_pre_election: true
I20260620 14:57:01.967571 25137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa9609774dd49fb87dcbcac217d528f" candidate_uuid: "6b5533003c01437dba728dd4180e8150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" is_pre_election: true
I20260620 14:57:01.967684 25268 raft_consensus.cc:2454] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b5533003c01437dba728dd4180e8150 in term 0.
I20260620 14:57:01.967882 25137 raft_consensus.cc:2454] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b5533003c01437dba728dd4180e8150 in term 0.
I20260620 14:57:01.968267 24942 leader_election.cc:302] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b5533003c01437dba728dd4180e8150, c8ee197997ca47929b90a868fa26ea98; no voters:
I20260620 14:57:01.968576 25338 raft_consensus.cc:2781] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:57:01.968662 25338 raft_consensus.cc:482] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:57:01.968713 25338 raft_consensus.cc:3037] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:01.969774 25338 raft_consensus.cc:504] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39097 } } peers { permanent_uuid: "c8ee197997ca47929b90a868fa26ea98" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45045 } } peers { permanent_uuid: "6b5533003c01437dba728dd4180e8150" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43955 } }
I20260620 14:57:01.970139 25338 leader_election.cc:288] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [CANDIDATE]: Term 1 election: Requested vote from peers 498f4760e0b343cea254a5dbfe5ccf3a (127.7.24.194:39097), c8ee197997ca47929b90a868fa26ea98 (127.7.24.195:45045)
I20260620 14:57:01.970556 25268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa9609774dd49fb87dcbcac217d528f" candidate_uuid: "6b5533003c01437dba728dd4180e8150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ee197997ca47929b90a868fa26ea98"
I20260620 14:57:01.970589 25137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa9609774dd49fb87dcbcac217d528f" candidate_uuid: "6b5533003c01437dba728dd4180e8150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498f4760e0b343cea254a5dbfe5ccf3a"
I20260620 14:57:01.970691 25268 raft_consensus.cc:3037] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:01.970724 25137 raft_consensus.cc:3037] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:01.971812 25137 raft_consensus.cc:2454] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b5533003c01437dba728dd4180e8150 in term 1.
I20260620 14:57:01.971953 25268 raft_consensus.cc:2454] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b5533003c01437dba728dd4180e8150 in term 1.
I20260620 14:57:01.972239 24939 leader_election.cc:302] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498f4760e0b343cea254a5dbfe5ccf3a, 6b5533003c01437dba728dd4180e8150; no voters:
I20260620 14:57:01.972460 25338 raft_consensus.cc:2781] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:01.972707 25338 raft_consensus.cc:686] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [term 1 LEADER]: Becoming Leader. State: Replica: 6b5533003c01437dba728dd4180e8150, State: Running, Role: LEADER
I20260620 14:57:01.973085 25338 consensus_queue.cc:237] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39097 } } peers { permanent_uuid: "c8ee197997ca47929b90a868fa26ea98" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45045 } } peers { permanent_uuid: "6b5533003c01437dba728dd4180e8150" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43955 } }
I20260620 14:57:01.976358 24864 catalog_manager.cc:5495] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b5533003c01437dba728dd4180e8150 (127.7.24.193). New cstate: current_term: 1 leader_uuid: "6b5533003c01437dba728dd4180e8150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8ee197997ca47929b90a868fa26ea98" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b5533003c01437dba728dd4180e8150" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43955 } health_report { overall_health: HEALTHY } } }
I20260620 14:57:02.009676 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:02.006033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.7.24.193:43955)
0620 14:57:02.006174 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:02.006178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:02.006196 (+ 18us) server_negotiation.cc:407] Connection header received
0620 14:57:02.006299 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:02.006302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:02.006366 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:02.006516 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:02.006789 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:02.007311 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:02.008161 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:02.008392 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:02.008507 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:02.008829 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:02.009265 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:02.009405 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:02.009467 (+ 62us) server_negotiation.cc:299] Negotiation successful
0620 14:57:02.009521 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 14:57:02.015693 25137 raft_consensus.cc:1261] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a [term 1 FOLLOWER]: Refusing update from remote peer 6b5533003c01437dba728dd4180e8150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:02.015695 25268 raft_consensus.cc:1261] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98 [term 1 FOLLOWER]: Refusing update from remote peer 6b5533003c01437dba728dd4180e8150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:02.016502 25342 consensus_queue.cc:1035] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "498f4760e0b343cea254a5dbfe5ccf3a" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:02.016696 25338 consensus_queue.cc:1035] T 1fa9609774dd49fb87dcbcac217d528f P 6b5533003c01437dba728dd4180e8150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8ee197997ca47929b90a868fa26ea98" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:02.023970 25347 mvcc.cc:205] Tried to move back new op lower bound from 7298938560568762368 to 7298938560404459520. Current Snapshot: MvccSnapshot[applied={T|T < 7298938560568762368}]
I20260620 14:57:02.032505 24864 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:02.032704 24864 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:02.035490 24864 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b918607b0ac4f2892d20d52fe4e1f74: Sending DeleteTablet for 3 replicas of tablet 1fa9609774dd49fb87dcbcac217d528f
I20260620 14:57:02.036329 25248 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fa9609774dd49fb87dcbcac217d528f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:02 UTC) from {username='slave'} at 127.0.0.1:52232
I20260620 14:57:02.036315 25117 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fa9609774dd49fb87dcbcac217d528f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:02 UTC) from {username='slave'} at 127.0.0.1:41888
I20260620 14:57:02.036499 24986 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fa9609774dd49fb87dcbcac217d528f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:02 UTC) from {username='slave'} at 127.0.0.1:42328
I20260620 14:57:02.037151 25355 tablet_replica.cc:330] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98: stopping tablet replica
I20260620 14:57:02.037166 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 24924
I20260620 14:57:02.038472 25356 tablet_replica.cc:330] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a: stopping tablet replica
I20260620 14:57:02.038765 25355 raft_consensus.cc:2229] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:57:02.038797 25356 raft_consensus.cc:2229] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:57:02.039075 25356 raft_consensus.cc:2258] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:02.039119 25355 raft_consensus.cc:2258] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:02.040449 25355 ts_tablet_manager.cc:1905] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:02.043324 25356 ts_tablet_manager.cc:1905] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:02.043738 25355 ts_tablet_manager.cc:1918] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:57:02.043865 25355 log.cc:1196] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/1fa9609774dd49fb87dcbcac217d528f
I20260620 14:57:02.044188 25355 ts_tablet_manager.cc:1939] T 1fa9609774dd49fb87dcbcac217d528f P c8ee197997ca47929b90a868fa26ea98: Deleting consensus metadata
W20260620 14:57:02.044554 24850 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:43955 (error 108)
I20260620 14:57:02.045502 24851 catalog_manager.cc:4841] TS c8ee197997ca47929b90a868fa26ea98 (127.7.24.195:45045): tablet 1fa9609774dd49fb87dcbcac217d528f (table test-table [id=01ff9172ec674ece93f50f176a90b719]) successfully deleted
I20260620 14:57:02.045934 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 25055
I20260620 14:57:02.045965 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:02.045679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:43955 (local address 127.0.0.1:42348)
0620 14:57:02.045814 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:02.045881 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:43955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 14:57:02.045974 25356 ts_tablet_manager.cc:1918] T 1fa9609774dd49fb87dcbcac217d528f P 498f4760e0b343cea254a5dbfe5ccf3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260620 14:57:02.046545 24850 catalog_manager.cc:4584] TS 6b5533003c01437dba728dd4180e8150 (127.7.24.193:43955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1fa9609774dd49fb87dcbcac217d528f: Network error: Client connection negotiation failed: client connection to 127.7.24.193:43955: connect: Connection refused (error 111)
I20260620 14:57:02.051856 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 25186
I20260620 14:57:02.052806 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:02.052087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:39097 (local address 127.0.0.1:41904)
0620 14:57:02.052632 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:02.052699 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.194:39097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444}
I20260620 14:57:02.057649 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 24833
2026-06-20T14:57:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1118 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:02 dist-test-slave-sms9 krb5kdc[25361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:02 dist-test-slave-sms9 krb5kdc[25361](info): set up 2 sockets
Jun 20 14:57:02 dist-test-slave-sms9 krb5kdc[25361](info): commencing operation
krb5kdc: starting...
W20260620 14:57:03.118409 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:03 dist-test-slave-sms9 krb5kdc[25361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:57:03.252007 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:36979
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:36979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:03.366575 25374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:03.366922 25374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:03.366997 25374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:03.371402 25374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:03.371524 25374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:03.371567 25374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:03.371608 25374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:03.371644 25374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:03.377537 25374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:36979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:36979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:03.378820 25374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:03.379940 25374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:03.388543 25382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:03.388651 25374 server_base.cc:1034] running on GCE node
W20260620 14:57:03.388554 25379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:03.388859 25380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:03.389698 25374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:03.390861 25374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:03.392058 25374 hybrid_clock.cc:648] HybridClock initialized: now 1781967423392022 us; error 54 us; skew 500 ppm
Jun 20 14:57:03 dist-test-slave-sms9 krb5kdc[25361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967423, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:03.394894 25374 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:57:03.395787 25374 webserver.cc:460] Webserver started at http://127.7.24.254:46413/ using document root <none> and password file <none>
I20260620 14:57:03.396363 25374 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:03.396523 25374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:03.396763 25374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:03.399042 25374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "32e954d5257345c6bedc02b62230cdfe"
format_stamp: "Formatted at 2026-06-20 14:57:03 on dist-test-slave-sms9"
server_key: "9973f8b0587291580a6c643842a4839b"
server_key_iv: "8bc4d62fa24f46dbe4da45faa8121458"
server_key_version: "encryptionkey@0"
I20260620 14:57:03.399608 25374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "32e954d5257345c6bedc02b62230cdfe"
format_stamp: "Formatted at 2026-06-20 14:57:03 on dist-test-slave-sms9"
server_key: "9973f8b0587291580a6c643842a4839b"
server_key_iv: "8bc4d62fa24f46dbe4da45faa8121458"
server_key_version: "encryptionkey@0"
I20260620 14:57:03.403324 25374 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 14:57:03.405864 25389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:03.407232 25374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 14:57:03.407389 25374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "32e954d5257345c6bedc02b62230cdfe"
format_stamp: "Formatted at 2026-06-20 14:57:03 on dist-test-slave-sms9"
server_key: "9973f8b0587291580a6c643842a4839b"
server_key_iv: "8bc4d62fa24f46dbe4da45faa8121458"
server_key_version: "encryptionkey@0"
I20260620 14:57:03.407518 25374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:03.417488 25374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:03.420826 25374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:03.421046 25374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:03.428261 25374 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:36979
I20260620 14:57:03.428272 25441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:36979 every 8 connection(s)
I20260620 14:57:03.429368 25374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:03.432331 25442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:03.438001 25442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe: Bootstrap starting.
I20260620 14:57:03.438926 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 25374
I20260620 14:57:03.439087 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:03.439437 7267 external_mini_cluster.cc:1428] Setting key b359d29a7258bb7220464e12688ea9b1
I20260620 14:57:03.441135 25442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:03.441900 25442 log.cc:824] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:03.444201 25442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe: No bootstrap required, opened a new log
I20260620 14:57:03.448415 25442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e954d5257345c6bedc02b62230cdfe" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36979 } }
I20260620 14:57:03.448674 25442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:03.448758 25442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e954d5257345c6bedc02b62230cdfe, State: Initialized, Role: FOLLOWER
I20260620 14:57:03.449214 25442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e954d5257345c6bedc02b62230cdfe" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36979 } }
I20260620 14:57:03.449373 25442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:03.449463 25442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:03.449572 25442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:03.450575 25442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e954d5257345c6bedc02b62230cdfe" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36979 } }
I20260620 14:57:03.450914 25442 leader_election.cc:302] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32e954d5257345c6bedc02b62230cdfe; no voters:
I20260620 14:57:03.451267 25442 leader_election.cc:288] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:03.451413 25447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:03.451665 25447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [term 1 LEADER]: Becoming Leader. State: Replica: 32e954d5257345c6bedc02b62230cdfe, State: Running, Role: LEADER
I20260620 14:57:03.452008 25447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e954d5257345c6bedc02b62230cdfe" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36979 } }
I20260620 14:57:03.452591 25442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:03.453366 25449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32e954d5257345c6bedc02b62230cdfe. Latest consensus state: current_term: 1 leader_uuid: "32e954d5257345c6bedc02b62230cdfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e954d5257345c6bedc02b62230cdfe" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36979 } } }
I20260620 14:57:03.453503 25449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [sys.catalog]: This master's current role is: LEADER
Jun 20 14:57:03 dist-test-slave-sms9 krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:03.453887 25448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32e954d5257345c6bedc02b62230cdfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e954d5257345c6bedc02b62230cdfe" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36979 } } }
I20260620 14:57:03.453971 25448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:03.454623 25456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:03.457893 25456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:03.461091 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:03.440719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38646 (local address 127.7.24.254:36979)
0620 14:57:03.441166 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:03.441181 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:03.441214 (+ 33us) server_negotiation.cc:407] Connection header received
0620 14:57:03.441906 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:03.441923 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:03.442371 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:03.442791 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:03.443746 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:03.444837 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:03.446009 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:03.446550 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:03.446668 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:03.454830 (+ 8162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:03.454839 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:03.454858 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:03.454895 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:03.457610 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:03.458202 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:03.458209 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:03.458214 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:03.458374 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:03.458745 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:03.458749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:03.458750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:03.459267 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:03.459479 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:03.459798 (+ 319us) server_negotiation.cc:299] Negotiation successful
0620 14:57:03.460108 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":128,"threads_started":1}
I20260620 14:57:03.463814 25456 catalog_manager.cc:1347] Generated new cluster ID: 3a5d51bb7c7c47e1befa3c888a35fbe4
I20260620 14:57:03.463889 25456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:03.477725 25456 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:03.478919 25456 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:03.484862 25456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe: Generated new TSK 0
I20260620 14:57:03.485566 25456 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:57:03.522722 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:36979
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:03.637689 25468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:03.638051 25468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:03.638099 25468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:03.642077 25468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:03.642138 25468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:03.642196 25468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:57:03.647454 25468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:36979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:03.648524 25468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:03.649487 25468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:03.659451 25474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:03.659466 25473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:03.659451 25476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:03.659775 25468 server_base.cc:1034] running on GCE node
I20260620 14:57:03.660179 25468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:03.660849 25468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:03.662021 25468 hybrid_clock.cc:648] HybridClock initialized: now 1781967423662004 us; error 29 us; skew 500 ppm
Jun 20 14:57:03 dist-test-slave-sms9 krb5kdc[25361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967423, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:03.664961 25468 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:57:03.665809 25468 webserver.cc:460] Webserver started at http://127.7.24.193:34869/ using document root <none> and password file <none>
I20260620 14:57:03.666329 25468 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:03.666396 25468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:03.666631 25468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:03.668697 25468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "5f15716ec56e42869ea33a5c5e17cb82"
format_stamp: "Formatted at 2026-06-20 14:57:03 on dist-test-slave-sms9"
server_key: "4de51dfb8d97bb67c4548f5a8fb069ce"
server_key_iv: "0f4cde25a608093ea3e91cc952d32722"
server_key_version: "encryptionkey@0"
I20260620 14:57:03.669234 25468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "5f15716ec56e42869ea33a5c5e17cb82"
format_stamp: "Formatted at 2026-06-20 14:57:03 on dist-test-slave-sms9"
server_key: "4de51dfb8d97bb67c4548f5a8fb069ce"
server_key_iv: "0f4cde25a608093ea3e91cc952d32722"
server_key_version: "encryptionkey@0"
I20260620 14:57:03.673255 25468 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:57:03.675813 25483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:03.677211 25468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 14:57:03.677388 25468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "5f15716ec56e42869ea33a5c5e17cb82"
format_stamp: "Formatted at 2026-06-20 14:57:03 on dist-test-slave-sms9"
server_key: "4de51dfb8d97bb67c4548f5a8fb069ce"
server_key_iv: "0f4cde25a608093ea3e91cc952d32722"
server_key_version: "encryptionkey@0"
I20260620 14:57:03.677521 25468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:03.697049 25468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:03.700644 25468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:03.700870 25468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:03.701498 25468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:03.702349 25468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:03.702422 25468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:03.702538 25468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:03.702582 25468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:03.712160 25468 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:33685
I20260620 14:57:03.712221 25596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:33685 every 8 connection(s)
I20260620 14:57:03.713176 25468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:57:03.719491 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 25468
I20260620 14:57:03.719684 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:57:03.720009 7267 external_mini_cluster.cc:1428] Setting key 67cf37d1a7bd914dee7ea570a59a43e4
Jun 20 14:57:03 dist-test-slave-sms9 krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967423, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:03.730046 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:03.715470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:39093 (local address 127.7.24.254:36979)
0620 14:57:03.715655 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:03.715659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:03.716582 (+ 923us) server_negotiation.cc:407] Connection header received
0620 14:57:03.717942 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:03.717947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:03.718007 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:03.718116 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:03.719654 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:03.720316 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:03.721746 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:03.722005 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:03.722122 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:03.725320 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:03.725327 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:03.725333 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:03.725368 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:03.727436 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:03.728068 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:03.728073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:03.728076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:03.728174 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:03.728587 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:03.728591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:03.728593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:03.728862 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:03.728986 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:03.729640 (+ 654us) server_negotiation.cc:299] Negotiation successful
0620 14:57:03.729793 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:57:03.730963 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:03.715880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:36979 (local address 127.7.24.193:39093)
0620 14:57:03.716401 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:03.716448 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:03.717709 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:03.718353 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:03.718361 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:03.718812 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:03.719427 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:03.719450 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:03.720475 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:03.720478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:03.721617 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:03.721627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:03.722166 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:03.722171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:03.722345 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:03.723128 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:03.723160 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:03.725103 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:03.727600 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:03.727606 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:03.727622 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:03.727943 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:03.728301 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:03.728304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:03.728305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:03.728474 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:03.729013 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:03.729019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:03.729341 (+ 322us) client_negotiation.cc:769] Sending connection context
0620 14:57:03.729544 (+ 203us) client_negotiation.cc:240] Negotiation successful
0620 14:57:03.729861 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":138,"threads_started":1}
I20260620 14:57:03.732551 25597 heartbeater.cc:347] Connected to a master server at 127.7.24.254:36979
I20260620 14:57:03.732846 25597 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:03.733465 25597 heartbeater.cc:511] Master 127.7.24.254:36979 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
I20260620 14:57:03.735172 25406 ts_manager.cc:194] Registered new tserver with Master: 5f15716ec56e42869ea33a5c5e17cb82 (127.7.24.193:33685)
I20260620 14:57:03.736937 25406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:39093
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:57:03.754326 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:36979
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:03.866250 25602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:03.866643 25602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:03.866736 25602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:03.870936 25602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:03.871021 25602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:03.871112 25602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:57:03.876642 25602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:36979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:03.877761 25602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:03.878940 25602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:03.888558 25608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:03.888587 25607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:03.888614 25610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:03.888900 25602 server_base.cc:1034] running on GCE node
I20260620 14:57:03.889369 25602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:03.890053 25602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:03.891230 25602 hybrid_clock.cc:648] HybridClock initialized: now 1781967423891208 us; error 35 us; skew 500 ppm
Jun 20 14:57:03 dist-test-slave-sms9 krb5kdc[25361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967423, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:03.894089 25602 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:57:03.895079 25602 webserver.cc:460] Webserver started at http://127.7.24.194:39293/ using document root <none> and password file <none>
I20260620 14:57:03.895668 25602 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:03.895761 25602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:03.895992 25602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:03.898046 25602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "a842a63a59644e0dad53a0b48d056fb3"
format_stamp: "Formatted at 2026-06-20 14:57:03 on dist-test-slave-sms9"
server_key: "95fdd53985bd82054e7bfc91538a9acf"
server_key_iv: "6c5971b95b0b41cca5c71d0c9084fed8"
server_key_version: "encryptionkey@0"
I20260620 14:57:03.898667 25602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "a842a63a59644e0dad53a0b48d056fb3"
format_stamp: "Formatted at 2026-06-20 14:57:03 on dist-test-slave-sms9"
server_key: "95fdd53985bd82054e7bfc91538a9acf"
server_key_iv: "6c5971b95b0b41cca5c71d0c9084fed8"
server_key_version: "encryptionkey@0"
I20260620 14:57:03.902551 25602 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260620 14:57:03.905018 25617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:03.906279 25602 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260620 14:57:03.906421 25602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "a842a63a59644e0dad53a0b48d056fb3"
format_stamp: "Formatted at 2026-06-20 14:57:03 on dist-test-slave-sms9"
server_key: "95fdd53985bd82054e7bfc91538a9acf"
server_key_iv: "6c5971b95b0b41cca5c71d0c9084fed8"
server_key_version: "encryptionkey@0"
I20260620 14:57:03.906600 25602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:03.933859 25602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:03.937095 25602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:03.937321 25602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:03.937935 25602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:03.938859 25602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:03.938932 25602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:03.939006 25602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:03.939059 25602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:03.948485 25602 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:42679
I20260620 14:57:03.948511 25730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:42679 every 8 connection(s)
I20260620 14:57:03.949386 25602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:57:03.951433 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 25602
I20260620 14:57:03.951558 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:57:03.951874 7267 external_mini_cluster.cc:1428] Setting key bfd7ff13af97a82f6451d6bb79a0b0e5
Jun 20 14:57:03 dist-test-slave-sms9 krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967423, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
I20260620 14:57:03.965951 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:03.951281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:37865 (local address 127.7.24.254:36979)
0620 14:57:03.951415 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:03.951419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:03.952485 (+ 1066us) server_negotiation.cc:407] Connection header received
0620 14:57:03.954350 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:03.954356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:03.954420 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:03.954537 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:03.956021 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:03.956738 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:03.958005 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:03.958402 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:03.958595 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:03.961426 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:03.961443 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:03.961448 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:03.961486 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:03.963221 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:03.963813 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:03.963848 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:03.963852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:03.963963 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:03.964420 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:03.964426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:03.964430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:03.964661 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:03.964776 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:03.965457 (+ 681us) server_negotiation.cc:299] Negotiation successful
0620 14:57:03.965615 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:57:03.966662 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:03.951794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:36979 (local address 127.7.24.194:37865)
0620 14:57:03.952317 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:03.952376 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:03.954114 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:03.954695 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:03.954707 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:03.955227 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:03.955855 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:03.955871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:03.956880 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:03.956883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:03.957868 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:03.957878 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:03.958624 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:03.958627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:03.958781 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:03.959422 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:03.959439 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:03.961173 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:03.963373 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:03.963379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:03.963396 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:03.963685 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:03.964097 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:03.964100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:03.964102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:03.964299 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:03.964820 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:03.964829 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:03.965168 (+ 339us) client_negotiation.cc:769] Sending connection context
0620 14:57:03.965355 (+ 187us) client_negotiation.cc:240] Negotiation successful
0620 14:57:03.965655 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":152,"threads_started":1}
I20260620 14:57:03.967944 25731 heartbeater.cc:347] Connected to a master server at 127.7.24.254:36979
I20260620 14:57:03.968214 25731 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:03.968734 25731 heartbeater.cc:511] Master 127.7.24.254:36979 requested a full tablet report, sending...
I20260620 14:57:03.969892 25406 ts_manager.cc:194] Registered new tserver with Master: a842a63a59644e0dad53a0b48d056fb3 (127.7.24.194:42679)
I20260620 14:57:03.970539 25406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:37865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:57:03.981998 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:36979
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:04.092156 25736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:04.092453 25736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:04.092523 25736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:04.096607 25736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:04.096693 25736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:04.096800 25736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:57:04.102013 25736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:36979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:04.103202 25736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:04.104264 25736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:04.114117 25744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:04.114228 25736 server_base.cc:1034] running on GCE node
W20260620 14:57:04.114171 25741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:04.114118 25742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:04.114949 25736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:04.115595 25736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:04.116751 25736 hybrid_clock.cc:648] HybridClock initialized: now 1781967424116737 us; error 27 us; skew 500 ppm
Jun 20 14:57:04 dist-test-slave-sms9 krb5kdc[25361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967424, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:04.119553 25736 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:57:04.120448 25736 webserver.cc:460] Webserver started at http://127.7.24.195:36271/ using document root <none> and password file <none>
I20260620 14:57:04.120968 25736 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:04.121026 25736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:04.121191 25736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:04.123231 25736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "052ddb2e06da4d8d99b7b2683b9da5a2"
format_stamp: "Formatted at 2026-06-20 14:57:04 on dist-test-slave-sms9"
server_key: "995c4eb9411d27328aabad3b2bcf28ff"
server_key_iv: "29379def859d38ce3fab28350728a10a"
server_key_version: "encryptionkey@0"
I20260620 14:57:04.123737 25736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "052ddb2e06da4d8d99b7b2683b9da5a2"
format_stamp: "Formatted at 2026-06-20 14:57:04 on dist-test-slave-sms9"
server_key: "995c4eb9411d27328aabad3b2bcf28ff"
server_key_iv: "29379def859d38ce3fab28350728a10a"
server_key_version: "encryptionkey@0"
I20260620 14:57:04.127379 25736 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:57:04.129779 25751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:04.131085 25736 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:57:04.131217 25736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "052ddb2e06da4d8d99b7b2683b9da5a2"
format_stamp: "Formatted at 2026-06-20 14:57:04 on dist-test-slave-sms9"
server_key: "995c4eb9411d27328aabad3b2bcf28ff"
server_key_iv: "29379def859d38ce3fab28350728a10a"
server_key_version: "encryptionkey@0"
I20260620 14:57:04.131328 25736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:04.157999 25736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:04.161433 25736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:04.161747 25736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:04.162376 25736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:04.163272 25736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:04.163347 25736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:04.163457 25736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:04.163503 25736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:04.172722 25736 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:39501
I20260620 14:57:04.172744 25864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:39501 every 8 connection(s)
I20260620 14:57:04.173744 25736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:57:04.178767 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 25736
I20260620 14:57:04.178912 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:57:04.179203 7267 external_mini_cluster.cc:1428] Setting key b37664936b370d18a081871101e502d5
Jun 20 14:57:04 dist-test-slave-sms9 krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967424, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:04.188359 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.175789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:42731 (local address 127.7.24.254:36979)
0620 14:57:04.175957 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:04.175961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:04.176899 (+ 938us) server_negotiation.cc:407] Connection header received
0620 14:57:04.178288 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:04.178292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:04.178365 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:04.178468 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:04.179819 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.180408 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.181867 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.182088 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.182200 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.184727 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:04.184730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:04.184732 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:04.184756 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:04.186227 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:04.186810 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:04.186815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:04.186817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:04.186898 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:04.187246 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:04.187250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:04.187251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:04.187438 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:04.187548 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:04.188042 (+ 494us) server_negotiation.cc:299] Negotiation successful
0620 14:57:04.188173 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260620 14:57:04.189129 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.176170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:36979 (local address 127.7.24.195:42731)
0620 14:57:04.176631 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:04.176674 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:04.178051 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:04.178674 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:04.178681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:04.179149 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:04.179680 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.179693 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.180538 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.180546 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.181751 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.181761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.182237 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.182241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.182411 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.183105 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:04.183121 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:04.184553 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:04.186366 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:04.186372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:04.186389 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:04.186711 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:04.186992 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:04.186996 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:04.186998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:04.187137 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:04.187545 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:04.187551 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:04.187800 (+ 249us) client_negotiation.cc:769] Sending connection context
0620 14:57:04.187985 (+ 185us) client_negotiation.cc:240] Negotiation successful
0620 14:57:04.188307 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":120,"threads_started":1}
I20260620 14:57:04.190162 25865 heartbeater.cc:347] Connected to a master server at 127.7.24.254:36979
I20260620 14:57:04.190379 25865 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:04.190899 25865 heartbeater.cc:511] Master 127.7.24.254:36979 requested a full tablet report, sending...
I20260620 14:57:04.192003 25406 ts_manager.cc:194] Registered new tserver with Master: 052ddb2e06da4d8d99b7b2683b9da5a2 (127.7.24.195:39501)
I20260620 14:57:04.192750 25406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:42731
I20260620 14:57:04.193374 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:57:04.203337 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.195448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.7.24.254:36979)
0620 14:57:04.195578 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:04.195581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:04.195745 (+ 164us) server_negotiation.cc:407] Connection header received
0620 14:57:04.195841 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:04.195843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:04.195883 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:04.195951 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:04.196661 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.197157 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.198030 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.198235 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.198313 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.199885 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:04.199888 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:04.199890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:04.199913 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:04.201251 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:04.201832 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:04.201835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:04.201836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:04.201913 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:04.202301 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:04.202304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:04.202306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:04.202622 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:04.202747 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:04.203049 (+ 302us) server_negotiation.cc:299] Negotiation successful
0620 14:57:04.203161 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 14:57:04.206753 25405 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:57:04.209738 25405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:57:04.225591 25879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.219639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.7.24.193:33685)
0620 14:57:04.220149 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:04.220166 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:04.220192 (+ 26us) server_negotiation.cc:407] Connection header received
0620 14:57:04.220270 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:04.220275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:04.220578 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:04.220766 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:04.221167 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.222316 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.224327 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.224887 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.224974 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.225090 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:04.225393 (+ 303us) server_negotiation.cc:299] Negotiation successful
0620 14:57:04.225474 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":113,"threads_started":1}
I20260620 14:57:04.225589 25876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.219250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33685 (local address 127.0.0.1:51060)
0620 14:57:04.219791 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:04.219804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:04.219868 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:04.220752 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:04.220757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:04.220778 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:04.221020 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.221029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.222453 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.222485 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.224162 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.224172 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.225025 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.225029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.225137 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.225174 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 14:57:04.225297 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 14:57:04.225425 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":195,"thread_start_us":135,"threads_started":1}
I20260620 14:57:04.226485 25880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.220247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:42679 (local address 127.0.0.1:38380)
0620 14:57:04.220498 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:04.220512 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:04.220599 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:04.221677 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:04.221680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:04.221699 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:04.221834 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.221841 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.223000 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.223004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.224169 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.224178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.226024 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.226028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.226130 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.226168 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 14:57:04.226233 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 14:57:04.226303 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":44,"thread_start_us":108,"threads_started":1}
I20260620 14:57:04.226861 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.219061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:39501 (local address 127.0.0.1:52902)
0620 14:57:04.219633 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:04.219691 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:04.219829 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:04.220824 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:04.220828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:04.220848 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:04.221049 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.221059 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.222345 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.222351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.224162 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.224172 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.226208 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.226211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.226316 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.226340 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 14:57:04.226418 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 14:57:04.226724 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":266,"thread_start_us":96,"threads_started":1}
I20260620 14:57:04.227023 25881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.220735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38380 (local address 127.7.24.194:42679)
0620 14:57:04.221235 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:04.221246 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:04.221265 (+ 19us) server_negotiation.cc:407] Connection header received
0620 14:57:04.221333 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:04.221337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:04.221521 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:04.221675 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:04.221960 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.222865 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.224301 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.224889 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.225030 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.225171 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:04.226761 (+ 1590us) server_negotiation.cc:299] Negotiation successful
0620 14:57:04.226867 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":154,"threads_started":1}
I20260620 14:57:04.227560 25878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.219230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52902 (local address 127.7.24.195:39501)
0620 14:57:04.220354 (+ 1124us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:04.220368 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:04.220395 (+ 27us) server_negotiation.cc:407] Connection header received
0620 14:57:04.220472 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:04.220480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:04.220662 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:04.221195 (+ 533us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:04.221326 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.222216 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.225236 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.226079 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.226194 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.226341 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:04.227340 (+ 999us) server_negotiation.cc:299] Negotiation successful
0620 14:57:04.227453 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":999,"thread_start_us":154,"threads_started":1}
I20260620 14:57:04.228999 25531 tablet_service.cc:1467] Processing CreateTablet for tablet 1f0fb68c05df4034a8f926fb825b58b1 (DEFAULT_TABLE table=test-table [id=70a0e1817f124e0f873ffc91b07932c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:04.229234 25665 tablet_service.cc:1467] Processing CreateTablet for tablet 1f0fb68c05df4034a8f926fb825b58b1 (DEFAULT_TABLE table=test-table [id=70a0e1817f124e0f873ffc91b07932c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:04.230099 25665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f0fb68c05df4034a8f926fb825b58b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:04.230118 25531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f0fb68c05df4034a8f926fb825b58b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:04.229987 25799 tablet_service.cc:1467] Processing CreateTablet for tablet 1f0fb68c05df4034a8f926fb825b58b1 (DEFAULT_TABLE table=test-table [id=70a0e1817f124e0f873ffc91b07932c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:04.230955 25799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f0fb68c05df4034a8f926fb825b58b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:04.237862 25882 tablet_bootstrap.cc:492] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: Bootstrap starting.
I20260620 14:57:04.238617 25883 tablet_bootstrap.cc:492] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82: Bootstrap starting.
I20260620 14:57:04.238782 25884 tablet_bootstrap.cc:492] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: Bootstrap starting.
I20260620 14:57:04.240401 25882 tablet_bootstrap.cc:654] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:04.240666 25883 tablet_bootstrap.cc:654] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:04.240666 25884 tablet_bootstrap.cc:654] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:04.241237 25882 log.cc:824] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:04.241557 25883 log.cc:824] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:04.241868 25884 log.cc:824] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:04.243523 25882 tablet_bootstrap.cc:492] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: No bootstrap required, opened a new log
I20260620 14:57:04.243749 25882 ts_tablet_manager.cc:1397] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:57:04.243774 25883 tablet_bootstrap.cc:492] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82: No bootstrap required, opened a new log
I20260620 14:57:04.243775 25884 tablet_bootstrap.cc:492] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: No bootstrap required, opened a new log
I20260620 14:57:04.243940 25884 ts_tablet_manager.cc:1397] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 14:57:04.243940 25883 ts_tablet_manager.cc:1397] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:57:04.246874 25882 raft_consensus.cc:348] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39501 } } peers { permanent_uuid: "5f15716ec56e42869ea33a5c5e17cb82" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33685 } } peers { permanent_uuid: "a842a63a59644e0dad53a0b48d056fb3" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 42679 } }
I20260620 14:57:04.247138 25882 raft_consensus.cc:374] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:04.247190 25882 raft_consensus.cc:729] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a842a63a59644e0dad53a0b48d056fb3, State: Initialized, Role: FOLLOWER
I20260620 14:57:04.247632 25882 consensus_queue.cc:260] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39501 } } peers { permanent_uuid: "5f15716ec56e42869ea33a5c5e17cb82" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33685 } } peers { permanent_uuid: "a842a63a59644e0dad53a0b48d056fb3" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 42679 } }
I20260620 14:57:04.248000 25884 raft_consensus.cc:348] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39501 } } peers { permanent_uuid: "5f15716ec56e42869ea33a5c5e17cb82" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33685 } } peers { permanent_uuid: "a842a63a59644e0dad53a0b48d056fb3" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 42679 } }
I20260620 14:57:04.248281 25884 raft_consensus.cc:374] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:04.248291 25731 heartbeater.cc:503] Master 127.7.24.254:36979 was elected leader, sending a full tablet report...
I20260620 14:57:04.248332 25884 raft_consensus.cc:729] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 052ddb2e06da4d8d99b7b2683b9da5a2, State: Initialized, Role: FOLLOWER
I20260620 14:57:04.248327 25883 raft_consensus.cc:348] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39501 } } peers { permanent_uuid: "5f15716ec56e42869ea33a5c5e17cb82" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33685 } } peers { permanent_uuid: "a842a63a59644e0dad53a0b48d056fb3" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 42679 } }
I20260620 14:57:04.248540 25883 raft_consensus.cc:374] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:04.248589 25883 raft_consensus.cc:729] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f15716ec56e42869ea33a5c5e17cb82, State: Initialized, Role: FOLLOWER
I20260620 14:57:04.248653 25882 ts_tablet_manager.cc:1428] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260620 14:57:04.248785 25884 consensus_queue.cc:260] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39501 } } peers { permanent_uuid: "5f15716ec56e42869ea33a5c5e17cb82" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33685 } } peers { permanent_uuid: "a842a63a59644e0dad53a0b48d056fb3" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 42679 } }
I20260620 14:57:04.248931 25883 consensus_queue.cc:260] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39501 } } peers { permanent_uuid: "5f15716ec56e42869ea33a5c5e17cb82" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33685 } } peers { permanent_uuid: "a842a63a59644e0dad53a0b48d056fb3" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 42679 } }
I20260620 14:57:04.249435 25865 heartbeater.cc:503] Master 127.7.24.254:36979 was elected leader, sending a full tablet report...
I20260620 14:57:04.249806 25884 ts_tablet_manager.cc:1428] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 14:57:04.249806 25883 ts_tablet_manager.cc:1428] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260620 14:57:04.250174 25597 heartbeater.cc:503] Master 127.7.24.254:36979 was elected leader, sending a full tablet report...
W20260620 14:57:04.424866 25866 tablet.cc:2369] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 14:57:04.450698 25732 tablet.cc:2369] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 14:57:04.464571 25598 tablet.cc:2369] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:57:04.592780 25889 raft_consensus.cc:482] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:57:04.592975 25889 raft_consensus.cc:504] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39501 } } peers { permanent_uuid: "5f15716ec56e42869ea33a5c5e17cb82" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33685 } } peers { permanent_uuid: "a842a63a59644e0dad53a0b48d056fb3" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 42679 } }
I20260620 14:57:04.594261 25889 leader_election.cc:288] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f15716ec56e42869ea33a5c5e17cb82 (127.7.24.193:33685), a842a63a59644e0dad53a0b48d056fb3 (127.7.24.194:42679)
I20260620 14:57:04.597759 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.594452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33685 (local address 127.7.24.195:55489)
0620 14:57:04.594610 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:04.594626 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:04.594739 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:04.595051 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:04.595053 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:04.595071 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:04.595192 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.595197 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.595925 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.595930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.596743 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.596752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.597416 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.597423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.597525 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.597545 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 14:57:04.597602 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 14:57:04.597654 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":20}
I20260620 14:57:04.597828 25879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.594666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:55489 (local address 127.7.24.193:33685)
0620 14:57:04.594815 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:04.594821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:04.594838 (+ 17us) server_negotiation.cc:407] Connection header received
0620 14:57:04.594899 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:04.594902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:04.594942 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:04.595038 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:04.595316 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.595816 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.596864 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.597268 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.597425 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.597485 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:04.597636 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 14:57:04.597707 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:57:04.598657 25891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.594883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:42679 (local address 127.7.24.195:38569)
0620 14:57:04.595240 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:04.595252 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:04.595371 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:04.595635 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:04.595638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:04.595652 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:04.595789 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.595795 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.596499 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.596504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.597314 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:04.597323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.598201 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.598204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:04.598316 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.598343 (+ 27us) client_negotiation.cc:769] Sending connection context
0620 14:57:04.598427 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 14:57:04.598517 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":19,"thread_start_us":139,"threads_started":1}
I20260620 14:57:04.598594 25551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f0fb68c05df4034a8f926fb825b58b1" candidate_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f15716ec56e42869ea33a5c5e17cb82" is_pre_election: true
I20260620 14:57:04.598685 25881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.594722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:38569 (local address 127.7.24.194:42679)
0620 14:57:04.594891 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:04.594895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:04.595340 (+ 445us) server_negotiation.cc:407] Connection header received
0620 14:57:04.595488 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:04.595491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:04.595540 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:04.595627 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:04.595905 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.596385 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.597462 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.598048 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.598185 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.598234 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:04.598523 (+ 289us) server_negotiation.cc:299] Negotiation successful
0620 14:57:04.598599 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260620 14:57:04.598904 25551 raft_consensus.cc:2454] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052ddb2e06da4d8d99b7b2683b9da5a2 in term 0.
I20260620 14:57:04.599237 25685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f0fb68c05df4034a8f926fb825b58b1" candidate_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a842a63a59644e0dad53a0b48d056fb3" is_pre_election: true
I20260620 14:57:04.599469 25752 leader_election.cc:302] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052ddb2e06da4d8d99b7b2683b9da5a2, 5f15716ec56e42869ea33a5c5e17cb82; no voters:
I20260620 14:57:04.599501 25685 raft_consensus.cc:2454] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 052ddb2e06da4d8d99b7b2683b9da5a2 in term 0.
I20260620 14:57:04.599756 25889 raft_consensus.cc:2781] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:57:04.599848 25889 raft_consensus.cc:482] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:57:04.599900 25889 raft_consensus.cc:3037] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:04.600901 25889 raft_consensus.cc:504] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39501 } } peers { permanent_uuid: "5f15716ec56e42869ea33a5c5e17cb82" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33685 } } peers { permanent_uuid: "a842a63a59644e0dad53a0b48d056fb3" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 42679 } }
I20260620 14:57:04.601229 25889 leader_election.cc:288] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [CANDIDATE]: Term 1 election: Requested vote from peers 5f15716ec56e42869ea33a5c5e17cb82 (127.7.24.193:33685), a842a63a59644e0dad53a0b48d056fb3 (127.7.24.194:42679)
I20260620 14:57:04.601517 25551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f0fb68c05df4034a8f926fb825b58b1" candidate_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f15716ec56e42869ea33a5c5e17cb82"
I20260620 14:57:04.601651 25551 raft_consensus.cc:3037] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:04.601666 25685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f0fb68c05df4034a8f926fb825b58b1" candidate_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a842a63a59644e0dad53a0b48d056fb3"
I20260620 14:57:04.601795 25685 raft_consensus.cc:3037] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:04.603015 25685 raft_consensus.cc:2454] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 052ddb2e06da4d8d99b7b2683b9da5a2 in term 1.
I20260620 14:57:04.603014 25551 raft_consensus.cc:2454] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 052ddb2e06da4d8d99b7b2683b9da5a2 in term 1.
I20260620 14:57:04.603338 25755 leader_election.cc:302] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 052ddb2e06da4d8d99b7b2683b9da5a2, a842a63a59644e0dad53a0b48d056fb3; no voters:
I20260620 14:57:04.603519 25889 raft_consensus.cc:2781] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:04.603765 25889 raft_consensus.cc:686] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 1 LEADER]: Becoming Leader. State: Replica: 052ddb2e06da4d8d99b7b2683b9da5a2, State: Running, Role: LEADER
I20260620 14:57:04.604085 25889 consensus_queue.cc:237] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39501 } } peers { permanent_uuid: "5f15716ec56e42869ea33a5c5e17cb82" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33685 } } peers { permanent_uuid: "a842a63a59644e0dad53a0b48d056fb3" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 42679 } }
I20260620 14:57:04.607357 25406 catalog_manager.cc:5495] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 052ddb2e06da4d8d99b7b2683b9da5a2 (127.7.24.195). New cstate: current_term: 1 leader_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "052ddb2e06da4d8d99b7b2683b9da5a2" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 39501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f15716ec56e42869ea33a5c5e17cb82" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a842a63a59644e0dad53a0b48d056fb3" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 42679 } health_report { overall_health: UNKNOWN } } }
I20260620 14:57:04.682040 25878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.678279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52912 (local address 127.7.24.195:39501)
0620 14:57:04.678415 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:04.678422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:04.678439 (+ 17us) server_negotiation.cc:407] Connection header received
0620 14:57:04.678583 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:04.678588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:04.678664 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:04.678796 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:04.679045 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.679556 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.680466 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.680675 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.680783 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.681075 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:04.681554 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:04.681671 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:04.681884 (+ 213us) server_negotiation.cc:299] Negotiation successful
0620 14:57:04.681929 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 14:57:04.687820 25551 raft_consensus.cc:1261] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82 [term 1 FOLLOWER]: Refusing update from remote peer 052ddb2e06da4d8d99b7b2683b9da5a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:04.687821 25685 raft_consensus.cc:1261] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3 [term 1 FOLLOWER]: Refusing update from remote peer 052ddb2e06da4d8d99b7b2683b9da5a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:04.688550 25892 consensus_queue.cc:1035] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a842a63a59644e0dad53a0b48d056fb3" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 42679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:04.688731 25889 consensus_queue.cc:1035] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f15716ec56e42869ea33a5c5e17cb82" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 33685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:04.696547 25895 mvcc.cc:205] Tried to move back new op lower bound from 7298938571514298368 to 7298938571180982272. Current Snapshot: MvccSnapshot[applied={T|T < 7298938571514298368}]
I20260620 14:57:04.696977 25897 mvcc.cc:205] Tried to move back new op lower bound from 7298938571514298368 to 7298938571180982272. Current Snapshot: MvccSnapshot[applied={T|T < 7298938571514298368}]
I20260620 14:57:04.705873 25402 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:04.706033 25402 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:04.708632 25402 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32e954d5257345c6bedc02b62230cdfe: Sending DeleteTablet for 3 replicas of tablet 1f0fb68c05df4034a8f926fb825b58b1
I20260620 14:57:04.709481 25799 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f0fb68c05df4034a8f926fb825b58b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:04 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:52902
I20260620 14:57:04.709476 25531 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f0fb68c05df4034a8f926fb825b58b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:04 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:51060
I20260620 14:57:04.709470 25665 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f0fb68c05df4034a8f926fb825b58b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:04 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:38380
I20260620 14:57:04.710031 25906 tablet_replica.cc:330] T 1f0fb68c05df4034a8f926fb825b58b1 P 5f15716ec56e42869ea33a5c5e17cb82: stopping tablet replica
I20260620 14:57:04.710641 25907 tablet_replica.cc:330] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: stopping tablet replica
I20260620 14:57:04.710955 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 25468
I20260620 14:57:04.711376 25905 tablet_replica.cc:330] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: stopping tablet replica
I20260620 14:57:04.711676 25905 raft_consensus.cc:2229] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:57:04.712045 25905 raft_consensus.cc:2258] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:04.712271 25907 raft_consensus.cc:2229] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:57:04.712630 25907 raft_consensus.cc:2258] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:04.713888 25905 ts_tablet_manager.cc:1905] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:04.715265 25907 ts_tablet_manager.cc:1905] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:04.716840 25905 ts_tablet_manager.cc:1918] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:57:04.717011 25905 log.cc:1196] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/1f0fb68c05df4034a8f926fb825b58b1
I20260620 14:57:04.717403 25905 ts_tablet_manager.cc:1939] T 1f0fb68c05df4034a8f926fb825b58b1 P a842a63a59644e0dad53a0b48d056fb3: Deleting consensus metadata
I20260620 14:57:04.718765 25393 catalog_manager.cc:4841] TS a842a63a59644e0dad53a0b48d056fb3 (127.7.24.194:42679): tablet 1f0fb68c05df4034a8f926fb825b58b1 (table test-table [id=70a0e1817f124e0f873ffc91b07932c3]) successfully deleted
I20260620 14:57:04.718849 25907 ts_tablet_manager.cc:1918] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:57:04.718940 25907 log.cc:1196] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/1f0fb68c05df4034a8f926fb825b58b1
I20260620 14:57:04.719246 25907 ts_tablet_manager.cc:1939] T 1f0fb68c05df4034a8f926fb825b58b1 P 052ddb2e06da4d8d99b7b2683b9da5a2: Deleting consensus metadata
I20260620 14:57:04.720463 25391 catalog_manager.cc:4841] TS 052ddb2e06da4d8d99b7b2683b9da5a2 (127.7.24.195:39501): tablet 1f0fb68c05df4034a8f926fb825b58b1 (table test-table [id=70a0e1817f124e0f873ffc91b07932c3]) successfully deleted
W20260620 14:57:04.720803 25390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:33685 (error 108)
I20260620 14:57:04.720950 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 25602
I20260620 14:57:04.727190 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.726839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:33685 (local address 127.0.0.1:51076)
0620 14:57:04.726998 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:04.727079 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:33685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 14:57:04.727264 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 25736
W20260620 14:57:04.727387 25390 catalog_manager.cc:4584] TS 5f15716ec56e42869ea33a5c5e17cb82 (127.7.24.193:33685): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1f0fb68c05df4034a8f926fb825b58b1: Network error: Client connection negotiation failed: client connection to 127.7.24.193:33685: connect: Connection refused (error 111)
I20260620 14:57:04.732416 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 25374
2026-06-20T14:57:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2675 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-20T14:57:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:04Z Disabled control of system clock
I20260620 14:57:04.763298 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:41649
--webserver_interface=127.7.24.254
--webserver_port=0
--builtin_ntp_servers=127.7.24.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:41649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 14:57:04.874233 25912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:04.874568 25912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:04.874614 25912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:04.879060 25912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:04.879133 25912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:04.879149 25912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:04.879166 25912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:04.879182 25912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:04.884063 25912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:41649
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:41649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:04.885119 25912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:04.886441 25912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:04.895759 25917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:04.895759 25918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:04.895776 25920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:04.896133 25912 server_base.cc:1034] running on GCE node
I20260620 14:57:04.896804 25912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:04.897984 25912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:04.899181 25912 hybrid_clock.cc:648] HybridClock initialized: now 1781967424899157 us; error 46 us; skew 500 ppm
I20260620 14:57:04.901139 25912 webserver.cc:460] Webserver started at http://127.7.24.254:43613/ using document root <none> and password file <none>
I20260620 14:57:04.901748 25912 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:04.901845 25912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:04.902081 25912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:04.904206 25912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "109fae2e0c3e4f5e82d96e1ef6a25253"
format_stamp: "Formatted at 2026-06-20 14:57:04 on dist-test-slave-sms9"
server_key: "742065fa1f148caf30c765511b69ebbf"
server_key_iv: "3cdcaf76c203adacd0b88f67add6166b"
server_key_version: "encryptionkey@0"
I20260620 14:57:04.904783 25912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "109fae2e0c3e4f5e82d96e1ef6a25253"
format_stamp: "Formatted at 2026-06-20 14:57:04 on dist-test-slave-sms9"
server_key: "742065fa1f148caf30c765511b69ebbf"
server_key_iv: "3cdcaf76c203adacd0b88f67add6166b"
server_key_version: "encryptionkey@0"
I20260620 14:57:04.908627 25912 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:57:04.911177 25926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:04.912535 25912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 14:57:04.912698 25912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "109fae2e0c3e4f5e82d96e1ef6a25253"
format_stamp: "Formatted at 2026-06-20 14:57:04 on dist-test-slave-sms9"
server_key: "742065fa1f148caf30c765511b69ebbf"
server_key_iv: "3cdcaf76c203adacd0b88f67add6166b"
server_key_version: "encryptionkey@0"
I20260620 14:57:04.912818 25912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:04.929023 25912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:04.929728 25912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:04.929934 25912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:04.937387 25978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:41649 every 8 connection(s)
I20260620 14:57:04.937387 25912 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:41649
I20260620 14:57:04.938450 25912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:04.939100 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 25912
I20260620 14:57:04.939257 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:04.939584 7267 external_mini_cluster.cc:1428] Setting key 5e0a4fd0353ea6851aed4f7b3143c195
I20260620 14:57:04.942067 25979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:04.947645 25979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253: Bootstrap starting.
I20260620 14:57:04.949102 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:04.941084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59138 (local address 127.7.24.254:41649)
0620 14:57:04.941712 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:04.941730 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:04.941764 (+ 34us) server_negotiation.cc:407] Connection header received
0620 14:57:04.942536 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:04.942554 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:04.942934 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:04.943330 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:04.943730 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.944689 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.945786 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:04.946234 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:04.946403 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:04.947335 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:04.947340 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:04.947352 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:04.947821 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:04.947894 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:04.947915 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:04.948021 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:04.948168 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 14:57:04.948417 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":130,"threads_started":1}
I20260620 14:57:04.950055 25979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:04.950901 25979 log.cc:824] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:04.952824 25979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253: No bootstrap required, opened a new log
I20260620 14:57:04.956415 25979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109fae2e0c3e4f5e82d96e1ef6a25253" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41649 } }
I20260620 14:57:04.956637 25979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:04.956700 25979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 109fae2e0c3e4f5e82d96e1ef6a25253, State: Initialized, Role: FOLLOWER
I20260620 14:57:04.957152 25979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109fae2e0c3e4f5e82d96e1ef6a25253" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41649 } }
I20260620 14:57:04.957311 25979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:04.957404 25979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:04.957528 25979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:04.958451 25979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109fae2e0c3e4f5e82d96e1ef6a25253" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41649 } }
I20260620 14:57:04.958841 25979 leader_election.cc:302] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 109fae2e0c3e4f5e82d96e1ef6a25253; no voters:
I20260620 14:57:04.959211 25979 leader_election.cc:288] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:04.959324 25984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:04.959570 25984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [term 1 LEADER]: Becoming Leader. State: Replica: 109fae2e0c3e4f5e82d96e1ef6a25253, State: Running, Role: LEADER
I20260620 14:57:04.959901 25984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109fae2e0c3e4f5e82d96e1ef6a25253" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41649 } }
I20260620 14:57:04.960412 25979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:04.961632 25986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 109fae2e0c3e4f5e82d96e1ef6a25253. Latest consensus state: current_term: 1 leader_uuid: "109fae2e0c3e4f5e82d96e1ef6a25253" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109fae2e0c3e4f5e82d96e1ef6a25253" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41649 } } }
I20260620 14:57:04.961642 25985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "109fae2e0c3e4f5e82d96e1ef6a25253" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109fae2e0c3e4f5e82d96e1ef6a25253" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41649 } } }
I20260620 14:57:04.961787 25986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:04.961791 25985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:04.962148 25993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:04.965284 25993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:04.970192 25993 catalog_manager.cc:1347] Generated new cluster ID: 365dba9b5e774eb2bcb706b0c0ccabec
I20260620 14:57:04.970255 25993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:04.984431 25993 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:04.985448 25993 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:04.999166 25993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253: Generated new TSK 0
I20260620 14:57:04.999883 25993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:05.012241 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:41649
--builtin_ntp_servers=127.7.24.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 14:57:05.124029 26003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:05.124284 26003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:05.124322 26003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:05.128445 26003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:05.128494 26003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:05.128574 26003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:57:05.133286 26003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:41649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:05.134371 26003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:05.135790 26003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:05.146199 26009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:05.146173 26011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:05.146203 26008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:05.146708 26003 server_base.cc:1034] running on GCE node
I20260620 14:57:05.147104 26003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:05.147720 26003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:05.148921 26003 hybrid_clock.cc:648] HybridClock initialized: now 1781967425148906 us; error 33 us; skew 500 ppm
I20260620 14:57:05.150898 26003 webserver.cc:460] Webserver started at http://127.7.24.193:42195/ using document root <none> and password file <none>
I20260620 14:57:05.151443 26003 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:05.151503 26003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:05.151672 26003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:05.153684 26003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "17fb11b4781b4192ab5315adfb172b69"
format_stamp: "Formatted at 2026-06-20 14:57:05 on dist-test-slave-sms9"
server_key: "255fa5aedcf8cb85a481bd6c58412246"
server_key_iv: "744f06df9ad54cc0b1a6cb48fa9161b0"
server_key_version: "encryptionkey@0"
I20260620 14:57:05.154285 26003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "17fb11b4781b4192ab5315adfb172b69"
format_stamp: "Formatted at 2026-06-20 14:57:05 on dist-test-slave-sms9"
server_key: "255fa5aedcf8cb85a481bd6c58412246"
server_key_iv: "744f06df9ad54cc0b1a6cb48fa9161b0"
server_key_version: "encryptionkey@0"
I20260620 14:57:05.158152 26003 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260620 14:57:05.160766 26017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:05.162173 26003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 14:57:05.162273 26003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "17fb11b4781b4192ab5315adfb172b69"
format_stamp: "Formatted at 2026-06-20 14:57:05 on dist-test-slave-sms9"
server_key: "255fa5aedcf8cb85a481bd6c58412246"
server_key_iv: "744f06df9ad54cc0b1a6cb48fa9161b0"
server_key_version: "encryptionkey@0"
I20260620 14:57:05.162441 26003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:05.176215 26003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:05.176980 26003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:05.177140 26003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:05.177685 26003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:05.178678 26003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:05.178730 26003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:05.178772 26003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:05.178797 26003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:05.187965 26003 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:32959
I20260620 14:57:05.187994 26130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:32959 every 8 connection(s)
I20260620 14:57:05.188922 26003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:57:05.197964 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.190767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:53447 (local address 127.7.24.254:41649)
0620 14:57:05.190913 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:05.190916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:05.191971 (+ 1055us) server_negotiation.cc:407] Connection header received
0620 14:57:05.193352 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:05.193355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:05.193402 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:05.193481 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:05.194388 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.194988 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.196178 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.196395 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.196475 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.197565 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:05.197568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:05.197569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:05.197581 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:05.197596 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:05.197603 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:05.197657 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:05.197780 (+ 123us) server_negotiation.cc:299] Negotiation successful
0620 14:57:05.197848 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 14:57:05.198691 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 26003
I20260620 14:57:05.198792 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:57:05.198916 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.191138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:41649 (local address 127.7.24.193:53447)
0620 14:57:05.191824 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:05.191866 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:05.193131 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:05.193672 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:05.193678 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:05.193771 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:05.194248 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.194261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.195116 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.195119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.196057 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.196067 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.196527 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.196530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.196662 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.197322 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:57:05.197324 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:05.197362 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:57:05.197423 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:05.197665 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:05.197669 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:05.197673 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 14:57:05.197759 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 14:57:05.197963 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":120,"threads_started":1}
I20260620 14:57:05.199136 7267 external_mini_cluster.cc:1428] Setting key 0f758f84f6d2e1af8eab9746726b086c
I20260620 14:57:05.200456 26131 heartbeater.cc:347] Connected to a master server at 127.7.24.254:41649
I20260620 14:57:05.200673 26131 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:05.201156 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:41649
--builtin_ntp_servers=127.7.24.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 14:57:05.201207 26131 heartbeater.cc:511] Master 127.7.24.254:41649 requested a full tablet report, sending...
I20260620 14:57:05.202877 25942 ts_manager.cc:194] Registered new tserver with Master: 17fb11b4781b4192ab5315adfb172b69 (127.7.24.193:32959)
I20260620 14:57:05.204062 25942 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.193:53447
W20260620 14:57:05.313890 26134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:05.314162 26134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:05.314201 26134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:05.318547 26134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:05.318611 26134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:05.318668 26134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:57:05.323578 26134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:41649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:05.324741 26134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:05.326150 26134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:05.335806 26139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:05.335816 26140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:05.335816 26142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:05.335870 26134 server_base.cc:1034] running on GCE node
I20260620 14:57:05.336519 26134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:05.337203 26134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:05.338394 26134 hybrid_clock.cc:648] HybridClock initialized: now 1781967425338383 us; error 32 us; skew 500 ppm
I20260620 14:57:05.340289 26134 webserver.cc:460] Webserver started at http://127.7.24.194:39015/ using document root <none> and password file <none>
I20260620 14:57:05.340884 26134 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:05.340981 26134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:05.341220 26134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:05.343371 26134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "df424739739e4a45aba99639df0fb1b5"
format_stamp: "Formatted at 2026-06-20 14:57:05 on dist-test-slave-sms9"
server_key: "a29e1f440525f91b1bb5b5c1523bde36"
server_key_iv: "16b4aba0f294100f5c79d59132d89ff5"
server_key_version: "encryptionkey@0"
I20260620 14:57:05.344048 26134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "df424739739e4a45aba99639df0fb1b5"
format_stamp: "Formatted at 2026-06-20 14:57:05 on dist-test-slave-sms9"
server_key: "a29e1f440525f91b1bb5b5c1523bde36"
server_key_iv: "16b4aba0f294100f5c79d59132d89ff5"
server_key_version: "encryptionkey@0"
I20260620 14:57:05.352242 26134 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260620 14:57:05.354895 26148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:05.356277 26134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 14:57:05.356419 26134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "df424739739e4a45aba99639df0fb1b5"
format_stamp: "Formatted at 2026-06-20 14:57:05 on dist-test-slave-sms9"
server_key: "a29e1f440525f91b1bb5b5c1523bde36"
server_key_iv: "16b4aba0f294100f5c79d59132d89ff5"
server_key_version: "encryptionkey@0"
I20260620 14:57:05.356573 26134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:05.371680 26134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:05.372334 26134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:05.372531 26134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:05.373106 26134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:05.374030 26134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:05.374104 26134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:05.374176 26134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:05.374228 26134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:05.384593 26134 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:46339
I20260620 14:57:05.384662 26261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:46339 every 8 connection(s)
I20260620 14:57:05.385540 26134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:57:05.391320 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 26134
I20260620 14:57:05.391476 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:57:05.391847 7267 external_mini_cluster.cc:1428] Setting key 88b4356e2f0fd331319f9feb7811f41c
I20260620 14:57:05.393980 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:41649
--builtin_ntp_servers=127.7.24.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 14:57:05.398979 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.387640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:38991 (local address 127.7.24.254:41649)
0620 14:57:05.387805 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:05.387810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:05.389125 (+ 1315us) server_negotiation.cc:407] Connection header received
0620 14:57:05.391537 (+ 2412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:05.391540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:05.391603 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:05.391715 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:05.393689 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.394525 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.396481 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.396690 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.396788 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.398306 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:05.398312 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:05.398318 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:05.398335 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:05.398362 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:05.398375 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:05.398478 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:05.398677 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 14:57:05.398760 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 14:57:05.400581 26264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.388038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:41649 (local address 127.7.24.194:38991)
0620 14:57:05.388908 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:05.388974 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:05.391164 (+ 2190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:05.391945 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:05.391957 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:05.392084 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:05.393523 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.393545 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.394668 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.394673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.396338 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.396355 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.396870 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.396877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.397125 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.397939 (+ 814us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:57:05.397943 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:05.398005 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:57:05.398101 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:05.398493 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:05.398503 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:05.398513 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 14:57:05.398660 (+ 147us) client_negotiation.cc:240] Negotiation successful
0620 14:57:05.398996 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":118,"threads_started":1}
I20260620 14:57:05.401976 26262 heartbeater.cc:347] Connected to a master server at 127.7.24.254:41649
I20260620 14:57:05.402271 26262 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:05.402988 26262 heartbeater.cc:511] Master 127.7.24.254:41649 requested a full tablet report, sending...
I20260620 14:57:05.404325 25942 ts_manager.cc:194] Registered new tserver with Master: df424739739e4a45aba99639df0fb1b5 (127.7.24.194:46339)
I20260620 14:57:05.405200 25942 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.194:38991
W20260620 14:57:05.510443 26265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:05.510782 26265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:05.510887 26265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:05.514935 26265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:05.515009 26265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:05.515096 26265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:57:05.520082 26265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:41649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:05.521174 26265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:05.522583 26265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:05.532102 26271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:05.532097 26270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:05.532106 26273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:05.532281 26265 server_base.cc:1034] running on GCE node
I20260620 14:57:05.532900 26265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:05.533635 26265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:05.534857 26265 hybrid_clock.cc:648] HybridClock initialized: now 1781967425534817 us; error 52 us; skew 500 ppm
I20260620 14:57:05.537014 26265 webserver.cc:460] Webserver started at http://127.7.24.195:33193/ using document root <none> and password file <none>
I20260620 14:57:05.537639 26265 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:05.537739 26265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:05.537986 26265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:05.540243 26265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "3b079e4e183a4d75a9115bff90567489"
format_stamp: "Formatted at 2026-06-20 14:57:05 on dist-test-slave-sms9"
server_key: "c5bac2d33c57980a87628e2f27a9808b"
server_key_iv: "60271583da2a260787af714db6cf34ec"
server_key_version: "encryptionkey@0"
I20260620 14:57:05.540926 26265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "3b079e4e183a4d75a9115bff90567489"
format_stamp: "Formatted at 2026-06-20 14:57:05 on dist-test-slave-sms9"
server_key: "c5bac2d33c57980a87628e2f27a9808b"
server_key_iv: "60271583da2a260787af714db6cf34ec"
server_key_version: "encryptionkey@0"
I20260620 14:57:05.544981 26265 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:57:05.547799 26279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:05.549163 26265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 14:57:05.549310 26265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "3b079e4e183a4d75a9115bff90567489"
format_stamp: "Formatted at 2026-06-20 14:57:05 on dist-test-slave-sms9"
server_key: "c5bac2d33c57980a87628e2f27a9808b"
server_key_iv: "60271583da2a260787af714db6cf34ec"
server_key_version: "encryptionkey@0"
I20260620 14:57:05.549448 26265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:05.572717 26265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:05.573454 26265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:05.573658 26265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:05.574270 26265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:05.575284 26265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:05.575363 26265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:05.575438 26265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:05.575481 26265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:05.584142 26265 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:32963
I20260620 14:57:05.584213 26392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:32963 every 8 connection(s)
I20260620 14:57:05.585089 26265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:57:05.591413 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 26265
I20260620 14:57:05.591568 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:57:05.591866 7267 external_mini_cluster.cc:1428] Setting key ef90e8f9167db220ad48a4050d83aaa1
I20260620 14:57:05.594573 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.586980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:58521 (local address 127.7.24.254:41649)
0620 14:57:05.587130 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:05.587134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:05.588166 (+ 1032us) server_negotiation.cc:407] Connection header received
0620 14:57:05.589547 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:05.589550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:05.589600 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:05.589677 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:05.590599 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.591128 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.592602 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.592897 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.592976 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.594077 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:05.594081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:05.594083 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:05.594096 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:05.594120 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:05.594137 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:05.594195 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:05.594388 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 14:57:05.594440 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 14:57:05.595394 26395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.587402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:41649 (local address 127.7.24.195:58521)
0620 14:57:05.588040 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:05.588081 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:05.589335 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:05.589841 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:05.589847 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:05.589941 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:05.590413 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.590425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.591263 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.591267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.592468 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.592478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.593059 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.593063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.593200 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.593813 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:57:05.593814 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:05.593867 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:57:05.593960 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:05.594221 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:05.594237 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:05.594241 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 14:57:05.594342 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 14:57:05.594614 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":117,"threads_started":1}
I20260620 14:57:05.596474 26393 heartbeater.cc:347] Connected to a master server at 127.7.24.254:41649
I20260620 14:57:05.596719 26393 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:05.597205 26393 heartbeater.cc:511] Master 127.7.24.254:41649 requested a full tablet report, sending...
I20260620 14:57:05.598140 25942 ts_manager.cc:194] Registered new tserver with Master: 3b079e4e183a4d75a9115bff90567489 (127.7.24.195:32963)
I20260620 14:57:05.598769 25942 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.195:58521
I20260620 14:57:05.606077 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:57:05.611840 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.608152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36130 (local address 127.7.24.254:41649)
0620 14:57:05.608307 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:05.608311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:05.608424 (+ 113us) server_negotiation.cc:407] Connection header received
0620 14:57:05.608491 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:05.608494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:05.608534 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:05.608609 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:05.608879 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.609387 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.610186 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.610428 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.610589 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.611348 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:05.611351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:05.611353 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:05.611366 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:05.611386 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:05.611394 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:05.611510 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:05.611594 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 14:57:05.611624 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 14:57:05.614951 25942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:57:05.618243 25942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:57:05.632944 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.626749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:32963 (local address 127.0.0.1:58074)
0620 14:57:05.627193 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:05.627245 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:05.627370 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:05.628120 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:05.628128 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:05.628155 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:05.628317 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.628324 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.629985 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.629991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.631464 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.631474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.632447 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.632451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.632564 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.632603 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 14:57:05.632718 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 14:57:05.632825 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":122,"thread_start_us":73,"threads_started":1}
I20260620 14:57:05.632958 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.626883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58074 (local address 127.7.24.195:32963)
0620 14:57:05.627599 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:05.627612 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:05.627638 (+ 26us) server_negotiation.cc:407] Connection header received
0620 14:57:05.627713 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:05.627718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:05.627925 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:05.628506 (+ 581us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:05.628653 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.629851 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.631566 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.632312 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.632424 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.632619 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:05.632742 (+ 123us) server_negotiation.cc:299] Negotiation successful
0620 14:57:05.632820 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":181,"threads_started":1}
I20260620 14:57:05.633749 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.627376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:46339 (local address 127.0.0.1:55244)
0620 14:57:05.628229 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:05.628241 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:05.628336 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:05.628757 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:05.628761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:05.628779 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:05.628865 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.628871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.630215 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.630218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.631662 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.631668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.633346 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.633349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.633445 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.633463 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 14:57:05.633553 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 14:57:05.633621 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":279,"thread_start_us":93,"threads_started":1}
I20260620 14:57:05.634037 26408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.627688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55244 (local address 127.7.24.194:46339)
0620 14:57:05.628086 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:05.628097 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:05.628351 (+ 254us) server_negotiation.cc:407] Connection header received
0620 14:57:05.628433 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:05.628440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:05.628617 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:05.628761 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:05.629006 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.630113 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.631779 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.632316 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.632444 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.632628 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:05.633588 (+ 960us) server_negotiation.cc:299] Negotiation successful
0620 14:57:05.633683 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":102,"threads_started":1}
I20260620 14:57:05.634371 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.626841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:32959 (local address 127.0.0.1:44784)
0620 14:57:05.627636 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:05.627650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:05.627742 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:05.628942 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:05.628945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:05.628962 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:05.629068 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.629073 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.630074 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.630080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.631351 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.631361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.633968 (+ 2607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.633971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.634067 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.634095 (+ 28us) client_negotiation.cc:769] Sending connection context
0620 14:57:05.634174 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 14:57:05.634238 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":43,"thread_start_us":80,"threads_started":1}
I20260620 14:57:05.634996 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.627699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44784 (local address 127.7.24.193:32959)
0620 14:57:05.628141 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:05.628153 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:05.628173 (+ 20us) server_negotiation.cc:407] Connection header received
0620 14:57:05.628583 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:05.628588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:05.628749 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:05.628888 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:05.629182 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.629956 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.631489 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.632312 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.632424 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.632619 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:05.634742 (+ 2123us) server_negotiation.cc:299] Negotiation successful
0620 14:57:05.634847 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":123,"threads_started":1}
I20260620 14:57:05.636143 26327 tablet_service.cc:1467] Processing CreateTablet for tablet e0f6cf130e2644d3acc21a57ff8bbb2e (DEFAULT_TABLE table=test-table [id=84fa257211c041eaa655422b03229c73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:05.636610 26196 tablet_service.cc:1467] Processing CreateTablet for tablet e0f6cf130e2644d3acc21a57ff8bbb2e (DEFAULT_TABLE table=test-table [id=84fa257211c041eaa655422b03229c73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:05.637234 26327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f6cf130e2644d3acc21a57ff8bbb2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:05.637542 26196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f6cf130e2644d3acc21a57ff8bbb2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:05.637431 26065 tablet_service.cc:1467] Processing CreateTablet for tablet e0f6cf130e2644d3acc21a57ff8bbb2e (DEFAULT_TABLE table=test-table [id=84fa257211c041eaa655422b03229c73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:05.638330 26065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f6cf130e2644d3acc21a57ff8bbb2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:05.644207 26410 tablet_bootstrap.cc:492] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489: Bootstrap starting.
I20260620 14:57:05.645205 26412 tablet_bootstrap.cc:492] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69: Bootstrap starting.
I20260620 14:57:05.646298 26410 tablet_bootstrap.cc:654] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:05.646816 26412 tablet_bootstrap.cc:654] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:05.647162 26410 log.cc:824] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:05.647751 26411 tablet_bootstrap.cc:492] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5: Bootstrap starting.
I20260620 14:57:05.648041 26412 log.cc:824] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:05.649264 26410 tablet_bootstrap.cc:492] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489: No bootstrap required, opened a new log
I20260620 14:57:05.649447 26410 ts_tablet_manager.cc:1397] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 14:57:05.649716 26411 tablet_bootstrap.cc:654] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:05.650184 26412 tablet_bootstrap.cc:492] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69: No bootstrap required, opened a new log
I20260620 14:57:05.650367 26412 ts_tablet_manager.cc:1397] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260620 14:57:05.650394 26411 log.cc:824] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:05.652311 26411 tablet_bootstrap.cc:492] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5: No bootstrap required, opened a new log
I20260620 14:57:05.652508 26411 ts_tablet_manager.cc:1397] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 14:57:05.653807 26410 raft_consensus.cc:348] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b079e4e183a4d75a9115bff90567489" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 32963 } } peers { permanent_uuid: "17fb11b4781b4192ab5315adfb172b69" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 32959 } } peers { permanent_uuid: "df424739739e4a45aba99639df0fb1b5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46339 } }
I20260620 14:57:05.654126 26410 raft_consensus.cc:374] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:05.654174 26410 raft_consensus.cc:729] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b079e4e183a4d75a9115bff90567489, State: Initialized, Role: FOLLOWER
I20260620 14:57:05.654049 26412 raft_consensus.cc:348] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b079e4e183a4d75a9115bff90567489" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 32963 } } peers { permanent_uuid: "17fb11b4781b4192ab5315adfb172b69" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 32959 } } peers { permanent_uuid: "df424739739e4a45aba99639df0fb1b5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46339 } }
I20260620 14:57:05.654289 26412 raft_consensus.cc:374] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:05.654340 26412 raft_consensus.cc:729] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fb11b4781b4192ab5315adfb172b69, State: Initialized, Role: FOLLOWER
I20260620 14:57:05.654733 26412 consensus_queue.cc:260] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b079e4e183a4d75a9115bff90567489" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 32963 } } peers { permanent_uuid: "17fb11b4781b4192ab5315adfb172b69" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 32959 } } peers { permanent_uuid: "df424739739e4a45aba99639df0fb1b5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46339 } }
I20260620 14:57:05.654733 26410 consensus_queue.cc:260] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b079e4e183a4d75a9115bff90567489" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 32963 } } peers { permanent_uuid: "17fb11b4781b4192ab5315adfb172b69" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 32959 } } peers { permanent_uuid: "df424739739e4a45aba99639df0fb1b5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46339 } }
I20260620 14:57:05.655637 26131 heartbeater.cc:503] Master 127.7.24.254:41649 was elected leader, sending a full tablet report...
I20260620 14:57:05.655673 26393 heartbeater.cc:503] Master 127.7.24.254:41649 was elected leader, sending a full tablet report...
I20260620 14:57:05.655761 26410 ts_tablet_manager.cc:1428] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260620 14:57:05.655958 26411 raft_consensus.cc:348] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b079e4e183a4d75a9115bff90567489" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 32963 } } peers { permanent_uuid: "17fb11b4781b4192ab5315adfb172b69" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 32959 } } peers { permanent_uuid: "df424739739e4a45aba99639df0fb1b5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46339 } }
I20260620 14:57:05.656179 26411 raft_consensus.cc:374] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:05.656224 26411 raft_consensus.cc:729] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df424739739e4a45aba99639df0fb1b5, State: Initialized, Role: FOLLOWER
I20260620 14:57:05.656615 26411 consensus_queue.cc:260] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b079e4e183a4d75a9115bff90567489" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 32963 } } peers { permanent_uuid: "17fb11b4781b4192ab5315adfb172b69" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 32959 } } peers { permanent_uuid: "df424739739e4a45aba99639df0fb1b5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46339 } }
I20260620 14:57:05.657106 26412 ts_tablet_manager.cc:1428] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260620 14:57:05.658102 26262 heartbeater.cc:503] Master 127.7.24.254:41649 was elected leader, sending a full tablet report...
I20260620 14:57:05.658459 26411 ts_tablet_manager.cc:1428] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260620 14:57:05.690095 26132 tablet.cc:2369] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:57:05.759409 26417 raft_consensus.cc:482] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:57:05.759644 26417 raft_consensus.cc:504] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b079e4e183a4d75a9115bff90567489" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 32963 } } peers { permanent_uuid: "17fb11b4781b4192ab5315adfb172b69" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 32959 } } peers { permanent_uuid: "df424739739e4a45aba99639df0fb1b5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46339 } }
I20260620 14:57:05.760926 26417 leader_election.cc:288] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b079e4e183a4d75a9115bff90567489 (127.7.24.195:32963), df424739739e4a45aba99639df0fb1b5 (127.7.24.194:46339)
I20260620 14:57:05.766534 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.761312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:34333 (local address 127.7.24.195:32963)
0620 14:57:05.761447 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:05.761452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:05.761743 (+ 291us) server_negotiation.cc:407] Connection header received
0620 14:57:05.761855 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:05.761859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:05.761913 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:05.762012 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:05.762439 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.763281 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.765176 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.765781 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.765936 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.765988 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:05.766296 (+ 308us) server_negotiation.cc:299] Negotiation successful
0620 14:57:05.766384 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:57:05.766588 26408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.761354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:41797 (local address 127.7.24.194:46339)
0620 14:57:05.761473 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:05.761479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:05.761806 (+ 327us) server_negotiation.cc:407] Connection header received
0620 14:57:05.761862 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:05.761865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:05.761913 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:05.762012 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:05.762442 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.763305 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.765176 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.765781 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.765936 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.765988 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:05.766375 (+ 387us) server_negotiation.cc:299] Negotiation successful
0620 14:57:05.766448 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 14:57:05.766562 26419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.761189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:32963 (local address 127.7.24.193:34333)
0620 14:57:05.761613 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:05.761630 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:05.761759 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:05.762040 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:05.762043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:05.762083 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:05.762330 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.762335 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.763411 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.763415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.765019 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.765031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.765942 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.765946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.766124 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.766170 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 14:57:05.766259 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 14:57:05.766349 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":263,"spinlock_wait_cycles":30080,"thread_start_us":91,"threads_started":1}
I20260620 14:57:05.766659 26420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.761189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:46339 (local address 127.7.24.193:41797)
0620 14:57:05.761701 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:05.761714 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:05.761786 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:05.762040 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:05.762043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:05.762084 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:05.762309 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.762319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.763411 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.763415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.765020 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:05.765031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.765942 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.765946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:05.766128 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.766171 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 14:57:05.766318 (+ 147us) client_negotiation.cc:240] Negotiation successful
0620 14:57:05.766367 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":329,"spinlock_wait_cycles":7168,"thread_start_us":87,"threads_started":1}
I20260620 14:57:05.767360 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f6cf130e2644d3acc21a57ff8bbb2e" candidate_uuid: "17fb11b4781b4192ab5315adfb172b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df424739739e4a45aba99639df0fb1b5" is_pre_election: true
I20260620 14:57:05.767360 26347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f6cf130e2644d3acc21a57ff8bbb2e" candidate_uuid: "17fb11b4781b4192ab5315adfb172b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b079e4e183a4d75a9115bff90567489" is_pre_election: true
I20260620 14:57:05.767692 26216 raft_consensus.cc:2454] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fb11b4781b4192ab5315adfb172b69 in term 0.
I20260620 14:57:05.767692 26347 raft_consensus.cc:2454] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fb11b4781b4192ab5315adfb172b69 in term 0.
I20260620 14:57:05.768198 26021 leader_election.cc:302] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fb11b4781b4192ab5315adfb172b69, 3b079e4e183a4d75a9115bff90567489; no voters:
I20260620 14:57:05.768513 26417 raft_consensus.cc:2781] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:57:05.768613 26417 raft_consensus.cc:482] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:57:05.768658 26417 raft_consensus.cc:3037] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:05.769691 26417 raft_consensus.cc:504] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b079e4e183a4d75a9115bff90567489" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 32963 } } peers { permanent_uuid: "17fb11b4781b4192ab5315adfb172b69" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 32959 } } peers { permanent_uuid: "df424739739e4a45aba99639df0fb1b5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46339 } }
I20260620 14:57:05.770035 26417 leader_election.cc:288] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [CANDIDATE]: Term 1 election: Requested vote from peers 3b079e4e183a4d75a9115bff90567489 (127.7.24.195:32963), df424739739e4a45aba99639df0fb1b5 (127.7.24.194:46339)
I20260620 14:57:05.770414 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f6cf130e2644d3acc21a57ff8bbb2e" candidate_uuid: "17fb11b4781b4192ab5315adfb172b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df424739739e4a45aba99639df0fb1b5"
I20260620 14:57:05.770567 26347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f6cf130e2644d3acc21a57ff8bbb2e" candidate_uuid: "17fb11b4781b4192ab5315adfb172b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b079e4e183a4d75a9115bff90567489"
I20260620 14:57:05.770619 26216 raft_consensus.cc:3037] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:05.770697 26347 raft_consensus.cc:3037] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:05.771718 26347 raft_consensus.cc:2454] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fb11b4781b4192ab5315adfb172b69 in term 1.
I20260620 14:57:05.771719 26216 raft_consensus.cc:2454] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fb11b4781b4192ab5315adfb172b69 in term 1.
I20260620 14:57:05.772069 26021 leader_election.cc:302] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fb11b4781b4192ab5315adfb172b69, 3b079e4e183a4d75a9115bff90567489; no voters:
I20260620 14:57:05.772231 26417 raft_consensus.cc:2781] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:05.772454 26417 raft_consensus.cc:686] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 1 LEADER]: Becoming Leader. State: Replica: 17fb11b4781b4192ab5315adfb172b69, State: Running, Role: LEADER
I20260620 14:57:05.772781 26417 consensus_queue.cc:237] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b079e4e183a4d75a9115bff90567489" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 32963 } } peers { permanent_uuid: "17fb11b4781b4192ab5315adfb172b69" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 32959 } } peers { permanent_uuid: "df424739739e4a45aba99639df0fb1b5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46339 } }
I20260620 14:57:05.775949 25941 catalog_manager.cc:5495] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17fb11b4781b4192ab5315adfb172b69 (127.7.24.193). New cstate: current_term: 1 leader_uuid: "17fb11b4781b4192ab5315adfb172b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b079e4e183a4d75a9115bff90567489" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 32963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17fb11b4781b4192ab5315adfb172b69" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 32959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df424739739e4a45aba99639df0fb1b5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46339 } health_report { overall_health: UNKNOWN } } }
I20260620 14:57:05.789070 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.785549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.7.24.193:32959)
0620 14:57:05.785674 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:05.785678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:05.785689 (+ 11us) server_negotiation.cc:407] Connection header received
0620 14:57:05.785750 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:05.785752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:05.785803 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:05.785918 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:05.786175 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.786740 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.787595 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:05.787809 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:05.787919 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:05.788224 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:05.788624 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:05.788781 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:05.788872 (+ 91us) server_negotiation.cc:299] Negotiation successful
0620 14:57:05.788907 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:57:05.794981 26216 raft_consensus.cc:1261] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5 [term 1 FOLLOWER]: Refusing update from remote peer 17fb11b4781b4192ab5315adfb172b69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:05.794982 26347 raft_consensus.cc:1261] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489 [term 1 FOLLOWER]: Refusing update from remote peer 17fb11b4781b4192ab5315adfb172b69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:05.795689 26417 consensus_queue.cc:1035] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df424739739e4a45aba99639df0fb1b5" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 46339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:05.795945 26421 consensus_queue.cc:1035] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b079e4e183a4d75a9115bff90567489" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 32963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:05.804024 26426 mvcc.cc:205] Tried to move back new op lower bound from 7298938576048906240 to 7298938575967858688. Current Snapshot: MvccSnapshot[applied={T|T < 7298938576048906240}]
I20260620 14:57:05.813177 25941 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:05.813386 25941 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:05.815949 25941 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 109fae2e0c3e4f5e82d96e1ef6a25253: Sending DeleteTablet for 3 replicas of tablet e0f6cf130e2644d3acc21a57ff8bbb2e
I20260620 14:57:05.816843 26065 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f6cf130e2644d3acc21a57ff8bbb2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:05 UTC) from {username='slave'} at 127.0.0.1:44784
I20260620 14:57:05.816877 26327 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f6cf130e2644d3acc21a57ff8bbb2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:05 UTC) from {username='slave'} at 127.0.0.1:58074
I20260620 14:57:05.817013 26196 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f6cf130e2644d3acc21a57ff8bbb2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:05 UTC) from {username='slave'} at 127.0.0.1:55244
I20260620 14:57:05.817466 26434 tablet_replica.cc:330] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69: stopping tablet replica
I20260620 14:57:05.817615 26436 tablet_replica.cc:330] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5: stopping tablet replica
I20260620 14:57:05.817855 26434 raft_consensus.cc:2229] T e0f6cf130e2644d3acc21a57ff8bbb2e P 17fb11b4781b4192ab5315adfb172b69 [term 1 LEADER]: Raft consensus shutting down.
I20260620 14:57:05.818037 26436 raft_consensus.cc:2229] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:57:05.818120 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 26003
I20260620 14:57:05.818962 26436 raft_consensus.cc:2258] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:05.822232 26435 tablet_replica.cc:330] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489: stopping tablet replica
I20260620 14:57:05.822642 26435 raft_consensus.cc:2229] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:57:05.822746 26436 ts_tablet_manager.cc:1905] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:05.823045 26435 raft_consensus.cc:2258] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:05.824280 26435 ts_tablet_manager.cc:1905] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:05.826273 26436 ts_tablet_manager.cc:1918] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:57:05.826390 26436 log.cc:1196] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/e0f6cf130e2644d3acc21a57ff8bbb2e
I20260620 14:57:05.826666 26435 ts_tablet_manager.cc:1918] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:57:05.826759 26436 ts_tablet_manager.cc:1939] T e0f6cf130e2644d3acc21a57ff8bbb2e P df424739739e4a45aba99639df0fb1b5: Deleting consensus metadata
I20260620 14:57:05.826845 26435 log.cc:1196] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/e0f6cf130e2644d3acc21a57ff8bbb2e
I20260620 14:57:05.827142 26435 ts_tablet_manager.cc:1939] T e0f6cf130e2644d3acc21a57ff8bbb2e P 3b079e4e183a4d75a9115bff90567489: Deleting consensus metadata
I20260620 14:57:05.828161 25930 catalog_manager.cc:4841] TS 3b079e4e183a4d75a9115bff90567489 (127.7.24.195:32963): tablet e0f6cf130e2644d3acc21a57ff8bbb2e (table test-table [id=84fa257211c041eaa655422b03229c73]) successfully deleted
I20260620 14:57:05.828431 25929 catalog_manager.cc:4841] TS df424739739e4a45aba99639df0fb1b5 (127.7.24.194:46339): tablet e0f6cf130e2644d3acc21a57ff8bbb2e (table test-table [id=84fa257211c041eaa655422b03229c73]) successfully deleted
W20260620 14:57:05.828608 25928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:32959 (error 108)
I20260620 14:57:05.828855 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 26134
I20260620 14:57:05.830546 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:05.830152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:32959 (local address 127.0.0.1:44794)
0620 14:57:05.830316 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:05.830415 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260620 14:57:05.830745 25928 catalog_manager.cc:4584] TS 17fb11b4781b4192ab5315adfb172b69 (127.7.24.193:32959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e0f6cf130e2644d3acc21a57ff8bbb2e: Network error: Client connection negotiation failed: client connection to 127.7.24.193:32959: connect: Connection refused (error 111)
I20260620 14:57:05.835342 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 26265
I20260620 14:57:05.841434 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 25912
2026-06-20T14:57:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1107 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:05 dist-test-slave-sms9 krb5kdc[26440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:05 dist-test-slave-sms9 krb5kdc[26440](info): set up 2 sockets
Jun 20 14:57:05 dist-test-slave-sms9 krb5kdc[26440](info): commencing operation
krb5kdc: starting...
W20260620 14:57:06.894428 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:06 dist-test-slave-sms9 krb5kdc[26440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:57:07.017906 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40493
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:40673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:40493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:07.127084 26453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:07.127398 26453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:07.127478 26453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:07.131552 26453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:07.131644 26453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:07.131681 26453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:07.131739 26453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:07.131774 26453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:07.137053 26453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:40673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:40493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:40493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:07.138163 26453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:07.139248 26453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:07.147536 26459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:07.147743 26453 server_base.cc:1034] running on GCE node
W20260620 14:57:07.147867 26461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:07.147756 26458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:07.149066 26453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:07.150404 26453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:07.151605 26453 hybrid_clock.cc:648] HybridClock initialized: now 1781967427151593 us; error 26 us; skew 500 ppm
Jun 20 14:57:07 dist-test-slave-sms9 krb5kdc[26440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967427, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:07.154305 26453 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:57:07.155251 26453 webserver.cc:460] Webserver started at http://127.7.24.254:33803/ using document root <none> and password file <none>
I20260620 14:57:07.155802 26453 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:07.155896 26453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:07.156131 26453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:07.158161 26453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "6b69c470953a402c90a42a5844d40624"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "6656f7a3fcb40bd2019aee15b06694b1"
server_key_iv: "b6d656201ad2d5b091a0bf9c83c00176"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.158749 26453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "6b69c470953a402c90a42a5844d40624"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "6656f7a3fcb40bd2019aee15b06694b1"
server_key_iv: "b6d656201ad2d5b091a0bf9c83c00176"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.162352 26453 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260620 14:57:07.164719 26468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:07.165918 26453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:57:07.166054 26453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "6b69c470953a402c90a42a5844d40624"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "6656f7a3fcb40bd2019aee15b06694b1"
server_key_iv: "b6d656201ad2d5b091a0bf9c83c00176"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.166165 26453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:07.184518 26453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:07.187884 26453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:07.188089 26453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:07.195092 26453 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:40493
I20260620 14:57:07.195117 26520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:40493 every 8 connection(s)
I20260620 14:57:07.196033 26453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:07.198693 26521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:07.203919 26521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624: Bootstrap starting.
I20260620 14:57:07.204454 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 26453
I20260620 14:57:07.204617 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:07.204890 7267 external_mini_cluster.cc:1428] Setting key 4c7cdd89d69e21f82bb0c43f9a4cbe9b
I20260620 14:57:07.206707 26521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:07.207585 26521 log.cc:824] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:07.209394 26521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624: No bootstrap required, opened a new log
I20260620 14:57:07.212314 26521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b69c470953a402c90a42a5844d40624" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40493 } }
I20260620 14:57:07.212555 26521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:07.212613 26521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b69c470953a402c90a42a5844d40624, State: Initialized, Role: FOLLOWER
I20260620 14:57:07.213068 26521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b69c470953a402c90a42a5844d40624" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40493 } }
I20260620 14:57:07.213188 26521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:07.213255 26521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:07.213378 26521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:07.214280 26521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b69c470953a402c90a42a5844d40624" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40493 } }
Jun 20 14:57:07 dist-test-slave-sms9 krb5kdc[26440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:07.214689 26521 leader_election.cc:302] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b69c470953a402c90a42a5844d40624; no voters:
I20260620 14:57:07.215086 26521 leader_election.cc:288] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:07.215179 26526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:07.215476 26526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [term 1 LEADER]: Becoming Leader. State: Replica: 6b69c470953a402c90a42a5844d40624, State: Running, Role: LEADER
I20260620 14:57:07.215814 26526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b69c470953a402c90a42a5844d40624" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40493 } }
I20260620 14:57:07.216487 26521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:07.219578 26528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b69c470953a402c90a42a5844d40624. Latest consensus state: current_term: 1 leader_uuid: "6b69c470953a402c90a42a5844d40624" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b69c470953a402c90a42a5844d40624" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40493 } } }
I20260620 14:57:07.219578 26527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b69c470953a402c90a42a5844d40624" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b69c470953a402c90a42a5844d40624" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 40493 } } }
I20260620 14:57:07.219748 26528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:07.219748 26527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:07.221095 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.206227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36122 (local address 127.7.24.254:40493)
0620 14:57:07.206679 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:07.206699 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:07.206741 (+ 42us) server_negotiation.cc:407] Connection header received
0620 14:57:07.207639 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:07.207663 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:07.208069 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:07.208401 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:07.209192 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.210219 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.211132 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.212104 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.212285 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.215140 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:07.215146 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:07.215163 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:07.215200 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:07.217466 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:07.217956 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:07.217962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:07.217968 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:07.218107 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:07.218417 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:07.218420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:07.218422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:07.218962 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:07.219123 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:07.219876 (+ 753us) server_negotiation.cc:299] Negotiation successful
0620 14:57:07.220180 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":125,"threads_started":1}
W20260620 14:57:07.222662 26540 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:57:07.222748 26540 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:57:07.222813 26542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:07.223531 26542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:07.229408 26542 catalog_manager.cc:1347] Generated new cluster ID: a111681711c742a98315e81751eeafee
I20260620 14:57:07.229481 26542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:07.244524 26542 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:07.245837 26542 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:07.251808 26542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624: Generated new TSK 0
I20260620 14:57:07.252612 26542 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.24.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
Entry for principal kudu/127.7.24.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab.
I20260620 14:57:07.286073 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40493
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:40673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:07.397396 26547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:07.397634 26547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:07.397675 26547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:07.401741 26547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:07.401789 26547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:07.401878 26547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:57:07.406936 26547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:40673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.193.keytab
--principal=kudu/127.7.24.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:07.407908 26547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:07.408903 26547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:07.418401 26552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:07.418408 26553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:07.418403 26555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:07.418617 26547 server_base.cc:1034] running on GCE node
I20260620 14:57:07.419039 26547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:07.419653 26547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:07.420850 26547 hybrid_clock.cc:648] HybridClock initialized: now 1781967427420819 us; error 42 us; skew 500 ppm
Jun 20 14:57:07 dist-test-slave-sms9 krb5kdc[26440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967427, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:07.423638 26547 init.cc:377] Logged in from keytab as kudu/127.7.24.193@KRBTEST.COM (short username kudu)
I20260620 14:57:07.424808 26547 webserver.cc:460] Webserver started at http://127.7.24.193:42301/ using document root <none> and password file <none>
I20260620 14:57:07.425385 26547 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:07.425479 26547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:07.425686 26547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:07.427753 26547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "13010ce8672042709dd35b226269d8a1"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "fff8936d87ee4168fa9fe9e2a974cf29"
server_key_iv: "397beb8cfce3e6740cfcc1258a25b0af"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.428313 26547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "13010ce8672042709dd35b226269d8a1"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "fff8936d87ee4168fa9fe9e2a974cf29"
server_key_iv: "397beb8cfce3e6740cfcc1258a25b0af"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.432014 26547 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:57:07.434643 26562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:07.436019 26547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:57:07.436187 26547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "13010ce8672042709dd35b226269d8a1"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "fff8936d87ee4168fa9fe9e2a974cf29"
server_key_iv: "397beb8cfce3e6740cfcc1258a25b0af"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.436338 26547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:07.477358 26547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:07.480762 26547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:07.480984 26547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:07.481611 26547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:07.482530 26547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:07.482604 26547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:07.482671 26547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:07.482703 26547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:07.491670 26547 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:34207
I20260620 14:57:07.491695 26675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:34207 every 8 connection(s)
I20260620 14:57:07.492740 26547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:57:07.503000 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 26547
I20260620 14:57:07.503149 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:57:07.503501 7267 external_mini_cluster.cc:1428] Setting key d5d2b947adc46b42d0b5c3c8835ee503
Jun 20 14:57:07 dist-test-slave-sms9 krb5kdc[26440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967427, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.193@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:07.510185 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.494772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:54379 (local address 127.7.24.254:40493)
0620 14:57:07.494910 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:07.494913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:07.496241 (+ 1328us) server_negotiation.cc:407] Connection header received
0620 14:57:07.497614 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:07.497618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:07.497681 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:07.497798 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:07.499527 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.500495 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.502546 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.503008 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.503132 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.506389 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:07.506393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:07.506395 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:07.506421 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:07.508065 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:07.508602 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:07.508607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:07.508609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:07.508697 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:07.509045 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:07.509050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:07.509052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:07.509250 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:07.509362 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:07.509886 (+ 524us) server_negotiation.cc:299] Negotiation successful
0620 14:57:07.510013 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:57:07.511154 26678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.495144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40493 (local address 127.7.24.193:54379)
0620 14:57:07.496048 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:07.496112 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:07.497332 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:07.498004 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:07.498015 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:07.498591 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:07.499334 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.499357 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.500670 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.500676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.502357 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.502370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.503346 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.503352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.503597 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.504332 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:07.504359 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:07.506216 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:07.508205 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:07.508211 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:07.508228 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:07.508493 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:07.508798 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:07.508801 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:07.508803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:07.508948 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:07.509361 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:07.509369 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:07.509626 (+ 257us) client_negotiation.cc:769] Sending connection context
0620 14:57:07.509832 (+ 206us) client_negotiation.cc:240] Negotiation successful
0620 14:57:07.510169 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":134,"threads_started":1}
I20260620 14:57:07.512462 26676 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40493
I20260620 14:57:07.512701 26676 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:07.513207 26676 heartbeater.cc:511] Master 127.7.24.254:40493 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.24.194@KRBTEST.COM; defaulting to no policy
I20260620 14:57:07.514858 26485 ts_manager.cc:194] Registered new tserver with Master: 13010ce8672042709dd35b226269d8a1 (127.7.24.193:34207)
I20260620 14:57:07.516211 26485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.193@KRBTEST.COM'} at 127.7.24.193:54379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
Entry for principal kudu/127.7.24.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab.
I20260620 14:57:07.532747 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40493
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:40673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:07.646121 26681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:07.646369 26681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:07.646413 26681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:07.650563 26681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:07.650617 26681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:07.650714 26681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:57:07.656059 26681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:40673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.194.keytab
--principal=kudu/127.7.24.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:07.657063 26681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:07.658066 26681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:07.667829 26687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:07.667945 26686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:07.667851 26689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:07.668416 26681 server_base.cc:1034] running on GCE node
I20260620 14:57:07.668871 26681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:07.669593 26681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:07.670809 26681 hybrid_clock.cc:648] HybridClock initialized: now 1781967427670792 us; error 28 us; skew 500 ppm
Jun 20 14:57:07 dist-test-slave-sms9 krb5kdc[26440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967427, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:07.673569 26681 init.cc:377] Logged in from keytab as kudu/127.7.24.194@KRBTEST.COM (short username kudu)
I20260620 14:57:07.674566 26681 webserver.cc:460] Webserver started at http://127.7.24.194:37813/ using document root <none> and password file <none>
I20260620 14:57:07.675134 26681 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:07.675238 26681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:07.675477 26681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:07.677489 26681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "12dd329c071d47358582fc9a836b4c5d"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "a1f4c18db3202726ebf32e355d20740f"
server_key_iv: "439b2ce4723cbba2b512f138a023e302"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.678048 26681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "12dd329c071d47358582fc9a836b4c5d"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "a1f4c18db3202726ebf32e355d20740f"
server_key_iv: "439b2ce4723cbba2b512f138a023e302"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.681975 26681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:57:07.684895 26696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:07.686231 26681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:57:07.686512 26681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "12dd329c071d47358582fc9a836b4c5d"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "a1f4c18db3202726ebf32e355d20740f"
server_key_iv: "439b2ce4723cbba2b512f138a023e302"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.686664 26681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:07.706593 26681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:07.709959 26681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:07.710186 26681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:07.710846 26681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:07.711748 26681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:07.711829 26681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:07.711903 26681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:07.711953 26681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:07.722811 26681 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:39095
I20260620 14:57:07.722847 26809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:39095 every 8 connection(s)
I20260620 14:57:07.723788 26681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:57:07.729864 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 26681
I20260620 14:57:07.729990 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:57:07.730293 7267 external_mini_cluster.cc:1428] Setting key 8bdeeba7990a0d0cc1d9041f770a5e25
Jun 20 14:57:07 dist-test-slave-sms9 krb5kdc[26440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967427, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.194@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:07.738986 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.725668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:53593 (local address 127.7.24.254:40493)
0620 14:57:07.725844 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:07.725850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:07.726749 (+ 899us) server_negotiation.cc:407] Connection header received
0620 14:57:07.727938 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:07.727941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:07.727991 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:07.728073 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:07.729189 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.729999 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.731420 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.731777 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.731888 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.735042 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:07.735045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:07.735047 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:07.735070 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:07.736827 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:07.737367 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:07.737375 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:07.737377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:07.737450 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:07.737798 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:07.737803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:07.737808 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:07.737996 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:07.738100 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:07.738681 (+ 581us) server_negotiation.cc:299] Negotiation successful
0620 14:57:07.738805 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 14:57:07.739919 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.726104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40493 (local address 127.7.24.194:53593)
0620 14:57:07.726607 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:07.726648 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:07.727749 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:07.728221 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:07.728227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:07.728616 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:07.729064 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.729076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.730139 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.730142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.731256 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.731265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.731961 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.731966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.732165 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.732853 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:07.732879 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:07.734867 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:07.736955 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:07.736961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:07.736978 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:07.737248 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:07.737544 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:07.737549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:07.737551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:07.737691 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:07.738110 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:07.738118 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:07.738371 (+ 253us) client_negotiation.cc:769] Sending connection context
0620 14:57:07.738609 (+ 238us) client_negotiation.cc:240] Negotiation successful
0620 14:57:07.738949 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for kudu/127.7.24.195@KRBTEST.COM; defaulting to no policy
I20260620 14:57:07.741259 26810 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40493
I20260620 14:57:07.741528 26810 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:07.742067 26810 heartbeater.cc:511] Master 127.7.24.254:40493 requested a full tablet report, sending...
I20260620 14:57:07.743194 26485 ts_manager.cc:194] Registered new tserver with Master: 12dd329c071d47358582fc9a836b4c5d (127.7.24.194:39095)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.195@KRBTEST.COM" created.
I20260620 14:57:07.744031 26485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.194@KRBTEST.COM'} at 127.7.24.194:53593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
Entry for principal kudu/127.7.24.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab.
I20260620 14:57:07.759711 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40493
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:40673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:07.876364 26815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:07.876621 26815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:07.876664 26815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:07.880816 26815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:07.880878 26815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:07.880968 26815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:57:07.886569 26815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:40673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.195.keytab
--principal=kudu/127.7.24.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:40493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:07.887663 26815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:07.888731 26815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:07.898286 26821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:07.898313 26820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:07.898373 26823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:07.898505 26815 server_base.cc:1034] running on GCE node
I20260620 14:57:07.898939 26815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:07.899587 26815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:07.900785 26815 hybrid_clock.cc:648] HybridClock initialized: now 1781967427900755 us; error 40 us; skew 500 ppm
Jun 20 14:57:07 dist-test-slave-sms9 krb5kdc[26440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967427, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:07.903746 26815 init.cc:377] Logged in from keytab as kudu/127.7.24.195@KRBTEST.COM (short username kudu)
I20260620 14:57:07.904623 26815 webserver.cc:460] Webserver started at http://127.7.24.195:38125/ using document root <none> and password file <none>
I20260620 14:57:07.905185 26815 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:07.905335 26815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:07.905573 26815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:07.908056 26815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "8820dd163a914008ab5826b445024e47"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "01c3a2bc2a388915162ceaa7dc3d7f23"
server_key_iv: "1427eded9db7521c2e6f7c89ee5d50fb"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.908634 26815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "8820dd163a914008ab5826b445024e47"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "01c3a2bc2a388915162ceaa7dc3d7f23"
server_key_iv: "1427eded9db7521c2e6f7c89ee5d50fb"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.912429 26815 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:57:07.915128 26830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:07.916504 26815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:57:07.916649 26815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "8820dd163a914008ab5826b445024e47"
format_stamp: "Formatted at 2026-06-20 14:57:07 on dist-test-slave-sms9"
server_key: "01c3a2bc2a388915162ceaa7dc3d7f23"
server_key_iv: "1427eded9db7521c2e6f7c89ee5d50fb"
server_key_version: "encryptionkey@0"
I20260620 14:57:07.916765 26815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:07.926380 26815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:07.929499 26815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:07.929711 26815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:07.930294 26815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:07.931190 26815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:07.931260 26815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:07.931326 26815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:07.931375 26815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:07.940240 26815 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:34911
I20260620 14:57:07.940383 26943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:34911 every 8 connection(s)
I20260620 14:57:07.941311 26815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:57:07.946920 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 26815
I20260620 14:57:07.947086 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:57:07.947443 7267 external_mini_cluster.cc:1428] Setting key 2be988960012a33f3c06c08df6175509
Jun 20 14:57:07 dist-test-slave-sms9 krb5kdc[26440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967427, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.195@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:07.954907 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.943201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:41945 (local address 127.7.24.254:40493)
0620 14:57:07.943389 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:07.943392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:07.944186 (+ 794us) server_negotiation.cc:407] Connection header received
0620 14:57:07.945053 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:07.945055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:07.945102 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:07.945174 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:07.946338 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.946988 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.948626 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.948913 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.949011 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.951631 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:07.951634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:07.951637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:07.951662 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:07.953099 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:07.953545 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:07.953547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:07.953548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:07.953619 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:07.953925 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:07.953928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:07.953930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:07.954096 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:07.954171 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:07.954686 (+ 515us) server_negotiation.cc:299] Negotiation successful
0620 14:57:07.954785 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 14:57:07.955724 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.943587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:40493 (local address 127.7.24.195:41945)
0620 14:57:07.944030 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:07.944070 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:07.944853 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:07.945371 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:07.945378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:07.945714 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:07.946180 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.946193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.947158 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.947163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.948520 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.948531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.949053 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.949056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.949214 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.949946 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:07.949962 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:07.951483 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:07.953199 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:07.953203 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:07.953214 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:07.953457 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:07.953696 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:07.953699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:07.953700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:07.953844 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:07.954184 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:07.954188 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:07.954422 (+ 234us) client_negotiation.cc:769] Sending connection context
0620 14:57:07.954642 (+ 220us) client_negotiation.cc:240] Negotiation successful
0620 14:57:07.954908 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":112,"threads_started":1}
I20260620 14:57:07.956872 26944 heartbeater.cc:347] Connected to a master server at 127.7.24.254:40493
I20260620 14:57:07.957108 26944 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:07.957558 26944 heartbeater.cc:511] Master 127.7.24.254:40493 requested a full tablet report, sending...
I20260620 14:57:07.958612 26485 ts_manager.cc:194] Registered new tserver with Master: 8820dd163a914008ab5826b445024e47 (127.7.24.195:34911)
I20260620 14:57:07.959185 26485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.24.195@KRBTEST.COM'} at 127.7.24.195:41945
I20260620 14:57:07.961519 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:57:07.971212 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.963562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36132 (local address 127.7.24.254:40493)
0620 14:57:07.963695 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:07.963699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:07.963869 (+ 170us) server_negotiation.cc:407] Connection header received
0620 14:57:07.963968 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:07.963971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:07.964008 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:07.964073 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:07.964756 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.965248 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.966081 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.966343 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.966434 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.967864 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:07.967867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:07.967869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:07.967891 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:07.969405 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:07.969878 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:07.969880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:07.969881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:07.969954 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:07.970186 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:07.970188 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:07.970189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:07.970553 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:07.970666 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:07.970927 (+ 261us) server_negotiation.cc:299] Negotiation successful
0620 14:57:07.971019 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 14:57:07.974167 26485 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:57:07.977509 26485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:57:07.992550 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.986605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.7.24.193:34207)
0620 14:57:07.987380 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:07.987399 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:07.987425 (+ 26us) server_negotiation.cc:407] Connection header received
0620 14:57:07.987503 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:07.987508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:07.987701 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:07.987869 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:07.988169 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.989369 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.991150 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.991889 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.992015 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.992167 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:07.992297 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 14:57:07.992394 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":97,"threads_started":1}
I20260620 14:57:07.992539 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.986211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:34207 (local address 127.0.0.1:44472)
0620 14:57:07.986748 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:07.986826 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:07.986954 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:07.987864 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:07.987870 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:07.987892 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:07.988040 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.988048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.989519 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.989522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.991013 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.991023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.992011 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.992015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.992118 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.992154 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 14:57:07.992259 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 14:57:07.992388 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":67,"thread_start_us":139,"threads_started":1}
I20260620 14:57:07.993144 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.986584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:34911 (local address 127.0.0.1:46446)
0620 14:57:07.988127 (+ 1543us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:07.988140 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:07.988223 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:07.988809 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:07.988811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:07.988825 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:07.988886 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.988890 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.990157 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.990160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.991465 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.991474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.992717 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.992721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.992820 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.992842 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 14:57:07.992919 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 14:57:07.992992 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1471,"mutex_wait_us":41,"thread_start_us":103,"threads_started":1}
I20260620 14:57:07.993485 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.987100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.7.24.194:39095)
0620 14:57:07.987406 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:07.987418 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:07.988005 (+ 587us) server_negotiation.cc:407] Connection header received
0620 14:57:07.988136 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:07.988143 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:07.988328 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:07.988479 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:07.988690 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.989781 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.991109 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.991947 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.992050 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.992183 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:07.992509 (+ 326us) server_negotiation.cc:299] Negotiation successful
0620 14:57:07.992766 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":117,"threads_started":1}
I20260620 14:57:07.993620 26960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.987483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46446 (local address 127.7.24.195:34911)
0620 14:57:07.987810 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:07.987820 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:07.988220 (+ 400us) server_negotiation.cc:407] Connection header received
0620 14:57:07.988332 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:07.988339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:07.988511 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:07.988652 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:07.988997 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.990003 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.991587 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.992426 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.992522 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.992647 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:07.993373 (+ 726us) server_negotiation.cc:299] Negotiation successful
0620 14:57:07.993477 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":155,"threads_started":1}
I20260620 14:57:07.993897 26956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:07.986265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:39095 (local address 127.0.0.1:44480)
0620 14:57:07.987891 (+ 1626us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:07.987911 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:07.988025 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:07.988462 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:07.988465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:07.988480 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:07.988596 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.988605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.989933 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.989936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.990984 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:07.990994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:07.992249 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:07.992255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:07.992343 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:07.992367 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 14:57:07.993673 (+ 1306us) client_negotiation.cc:240] Negotiation successful
0620 14:57:07.993767 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1565,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260620 14:57:07.995764 26610 tablet_service.cc:1467] Processing CreateTablet for tablet e9ab92907295404ea95aec71dbc2f9fd (DEFAULT_TABLE table=test-table [id=b5596f32213d405a8d52f31c8b3789de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:07.996472 26744 tablet_service.cc:1467] Processing CreateTablet for tablet e9ab92907295404ea95aec71dbc2f9fd (DEFAULT_TABLE table=test-table [id=b5596f32213d405a8d52f31c8b3789de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:07.996928 26610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ab92907295404ea95aec71dbc2f9fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:07.997390 26744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ab92907295404ea95aec71dbc2f9fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:07.997936 26878 tablet_service.cc:1467] Processing CreateTablet for tablet e9ab92907295404ea95aec71dbc2f9fd (DEFAULT_TABLE table=test-table [id=b5596f32213d405a8d52f31c8b3789de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:07.998754 26878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ab92907295404ea95aec71dbc2f9fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:08.004216 26962 tablet_bootstrap.cc:492] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1: Bootstrap starting.
I20260620 14:57:08.004644 26961 tablet_bootstrap.cc:492] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47: Bootstrap starting.
I20260620 14:57:08.005043 26963 tablet_bootstrap.cc:492] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d: Bootstrap starting.
I20260620 14:57:08.007473 26962 tablet_bootstrap.cc:654] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:08.007977 26963 tablet_bootstrap.cc:654] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:08.008277 26961 tablet_bootstrap.cc:654] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:08.008813 26963 log.cc:824] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:08.008813 26962 log.cc:824] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:08.009279 26961 log.cc:824] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:08.011041 26962 tablet_bootstrap.cc:492] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1: No bootstrap required, opened a new log
I20260620 14:57:08.011041 26963 tablet_bootstrap.cc:492] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d: No bootstrap required, opened a new log
I20260620 14:57:08.011247 26963 ts_tablet_manager.cc:1397] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 14:57:08.011242 26962 ts_tablet_manager.cc:1397] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260620 14:57:08.012025 26961 tablet_bootstrap.cc:492] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47: No bootstrap required, opened a new log
I20260620 14:57:08.012231 26961 ts_tablet_manager.cc:1397] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260620 14:57:08.015311 26962 raft_consensus.cc:348] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13010ce8672042709dd35b226269d8a1" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34207 } } peers { permanent_uuid: "12dd329c071d47358582fc9a836b4c5d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39095 } } peers { permanent_uuid: "8820dd163a914008ab5826b445024e47" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34911 } }
I20260620 14:57:08.015311 26963 raft_consensus.cc:348] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13010ce8672042709dd35b226269d8a1" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34207 } } peers { permanent_uuid: "12dd329c071d47358582fc9a836b4c5d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39095 } } peers { permanent_uuid: "8820dd163a914008ab5826b445024e47" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34911 } }
I20260620 14:57:08.015599 26963 raft_consensus.cc:374] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:08.015599 26962 raft_consensus.cc:374] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:08.015659 26963 raft_consensus.cc:729] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12dd329c071d47358582fc9a836b4c5d, State: Initialized, Role: FOLLOWER
I20260620 14:57:08.015659 26962 raft_consensus.cc:729] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13010ce8672042709dd35b226269d8a1, State: Initialized, Role: FOLLOWER
I20260620 14:57:08.015558 26961 raft_consensus.cc:348] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13010ce8672042709dd35b226269d8a1" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34207 } } peers { permanent_uuid: "12dd329c071d47358582fc9a836b4c5d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39095 } } peers { permanent_uuid: "8820dd163a914008ab5826b445024e47" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34911 } }
I20260620 14:57:08.015873 26961 raft_consensus.cc:374] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:08.015950 26961 raft_consensus.cc:729] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8820dd163a914008ab5826b445024e47, State: Initialized, Role: FOLLOWER
I20260620 14:57:08.016173 26963 consensus_queue.cc:260] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13010ce8672042709dd35b226269d8a1" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34207 } } peers { permanent_uuid: "12dd329c071d47358582fc9a836b4c5d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39095 } } peers { permanent_uuid: "8820dd163a914008ab5826b445024e47" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34911 } }
I20260620 14:57:08.016173 26962 consensus_queue.cc:260] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13010ce8672042709dd35b226269d8a1" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34207 } } peers { permanent_uuid: "12dd329c071d47358582fc9a836b4c5d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39095 } } peers { permanent_uuid: "8820dd163a914008ab5826b445024e47" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34911 } }
I20260620 14:57:08.016425 26961 consensus_queue.cc:260] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13010ce8672042709dd35b226269d8a1" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34207 } } peers { permanent_uuid: "12dd329c071d47358582fc9a836b4c5d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39095 } } peers { permanent_uuid: "8820dd163a914008ab5826b445024e47" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34911 } }
I20260620 14:57:08.016868 26676 heartbeater.cc:503] Master 127.7.24.254:40493 was elected leader, sending a full tablet report...
I20260620 14:57:08.017170 26963 ts_tablet_manager.cc:1428] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 14:57:08.017452 26961 ts_tablet_manager.cc:1428] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260620 14:57:08.017820 26944 heartbeater.cc:503] Master 127.7.24.254:40493 was elected leader, sending a full tablet report...
I20260620 14:57:08.017921 26810 heartbeater.cc:503] Master 127.7.24.254:40493 was elected leader, sending a full tablet report...
I20260620 14:57:08.019018 26962 ts_tablet_manager.cc:1428] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260620 14:57:08.081696 26967 raft_consensus.cc:482] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:57:08.081964 26967 raft_consensus.cc:504] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13010ce8672042709dd35b226269d8a1" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34207 } } peers { permanent_uuid: "12dd329c071d47358582fc9a836b4c5d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39095 } } peers { permanent_uuid: "8820dd163a914008ab5826b445024e47" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34911 } }
I20260620 14:57:08.083276 26967 leader_election.cc:288] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12dd329c071d47358582fc9a836b4c5d (127.7.24.194:39095), 8820dd163a914008ab5826b445024e47 (127.7.24.195:34911)
I20260620 14:57:08.088969 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.083631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:43089 (local address 127.7.24.194:39095)
0620 14:57:08.083778 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:08.083783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:08.084049 (+ 266us) server_negotiation.cc:407] Connection header received
0620 14:57:08.084205 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:08.084209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:08.084264 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:08.084342 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:08.084814 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.085623 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.087721 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.088400 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.088504 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:08.088565 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:08.088774 (+ 209us) server_negotiation.cc:299] Negotiation successful
0620 14:57:08.088867 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 14:57:08.088999 26960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.083698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:38123 (local address 127.7.24.195:34911)
0620 14:57:08.083896 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:08.083900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:08.084091 (+ 191us) server_negotiation.cc:407] Connection header received
0620 14:57:08.084159 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:08.084164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:08.084225 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:08.084337 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:08.084814 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.085657 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.087614 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.088301 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.088444 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:08.088508 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:08.088791 (+ 283us) server_negotiation.cc:299] Negotiation successful
0620 14:57:08.088873 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260620 14:57:08.089021 26970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.083530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:39095 (local address 127.7.24.193:43089)
0620 14:57:08.083909 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:08.083925 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:08.084064 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:08.084411 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:08.084415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:08.084444 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:08.084634 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:08.084641 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.085806 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.085810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:08.087486 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:08.087494 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.088505 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.088508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:08.088597 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:08.088645 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 14:57:08.088735 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 14:57:08.088838 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":257,"spinlock_wait_cycles":22272,"thread_start_us":134,"threads_started":1}
I20260620 14:57:08.089394 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.083586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:34911 (local address 127.7.24.193:38123)
0620 14:57:08.083909 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:08.083925 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:08.084064 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:08.084411 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:08.084415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:08.084444 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:08.084634 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:08.084641 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.085806 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.085810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:08.087483 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:08.087494 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.088467 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.088475 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:08.088592 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:08.088673 (+ 81us) client_negotiation.cc:769] Sending connection context
0620 14:57:08.089151 (+ 478us) client_negotiation.cc:240] Negotiation successful
0620 14:57:08.089239 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":201,"spinlock_wait_cycles":21120,"thread_start_us":115,"threads_started":1}
I20260620 14:57:08.089941 26764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ab92907295404ea95aec71dbc2f9fd" candidate_uuid: "13010ce8672042709dd35b226269d8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12dd329c071d47358582fc9a836b4c5d" is_pre_election: true
I20260620 14:57:08.089982 26898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ab92907295404ea95aec71dbc2f9fd" candidate_uuid: "13010ce8672042709dd35b226269d8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8820dd163a914008ab5826b445024e47" is_pre_election: true
I20260620 14:57:08.090265 26898 raft_consensus.cc:2454] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13010ce8672042709dd35b226269d8a1 in term 0.
I20260620 14:57:08.090265 26764 raft_consensus.cc:2454] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13010ce8672042709dd35b226269d8a1 in term 0.
I20260620 14:57:08.090850 26566 leader_election.cc:302] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 12dd329c071d47358582fc9a836b4c5d, 13010ce8672042709dd35b226269d8a1, 8820dd163a914008ab5826b445024e47; no voters:
I20260620 14:57:08.091132 26967 raft_consensus.cc:2781] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:57:08.091216 26967 raft_consensus.cc:482] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:57:08.091259 26967 raft_consensus.cc:3037] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:08.092268 26967 raft_consensus.cc:504] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13010ce8672042709dd35b226269d8a1" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34207 } } peers { permanent_uuid: "12dd329c071d47358582fc9a836b4c5d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39095 } } peers { permanent_uuid: "8820dd163a914008ab5826b445024e47" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34911 } }
I20260620 14:57:08.092631 26967 leader_election.cc:288] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [CANDIDATE]: Term 1 election: Requested vote from peers 12dd329c071d47358582fc9a836b4c5d (127.7.24.194:39095), 8820dd163a914008ab5826b445024e47 (127.7.24.195:34911)
I20260620 14:57:08.092976 26764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ab92907295404ea95aec71dbc2f9fd" candidate_uuid: "13010ce8672042709dd35b226269d8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12dd329c071d47358582fc9a836b4c5d"
I20260620 14:57:08.093079 26898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ab92907295404ea95aec71dbc2f9fd" candidate_uuid: "13010ce8672042709dd35b226269d8a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8820dd163a914008ab5826b445024e47"
I20260620 14:57:08.093098 26764 raft_consensus.cc:3037] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:08.093238 26898 raft_consensus.cc:3037] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:08.094161 26764 raft_consensus.cc:2454] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13010ce8672042709dd35b226269d8a1 in term 1.
I20260620 14:57:08.094385 26898 raft_consensus.cc:2454] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13010ce8672042709dd35b226269d8a1 in term 1.
I20260620 14:57:08.094516 26565 leader_election.cc:302] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12dd329c071d47358582fc9a836b4c5d, 13010ce8672042709dd35b226269d8a1; no voters:
I20260620 14:57:08.094684 26967 raft_consensus.cc:2781] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:08.094883 26967 raft_consensus.cc:686] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 13010ce8672042709dd35b226269d8a1, State: Running, Role: LEADER
I20260620 14:57:08.095233 26967 consensus_queue.cc:237] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13010ce8672042709dd35b226269d8a1" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34207 } } peers { permanent_uuid: "12dd329c071d47358582fc9a836b4c5d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39095 } } peers { permanent_uuid: "8820dd163a914008ab5826b445024e47" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34911 } }
I20260620 14:57:08.098353 26482 catalog_manager.cc:5495] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13010ce8672042709dd35b226269d8a1 (127.7.24.193). New cstate: current_term: 1 leader_uuid: "13010ce8672042709dd35b226269d8a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13010ce8672042709dd35b226269d8a1" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12dd329c071d47358582fc9a836b4c5d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8820dd163a914008ab5826b445024e47" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260620 14:57:08.121459 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.117668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44488 (local address 127.7.24.193:34207)
0620 14:57:08.117801 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:08.117806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:08.117821 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:57:08.117973 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:08.117976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:08.118034 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:08.118124 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:08.118386 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.118970 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.119873 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.120093 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.120185 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:08.120549 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:08.121023 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:08.121119 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:08.121239 (+ 120us) server_negotiation.cc:299] Negotiation successful
0620 14:57:08.121299 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 14:57:08.127300 26764 raft_consensus.cc:1261] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d [term 1 FOLLOWER]: Refusing update from remote peer 13010ce8672042709dd35b226269d8a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:08.127406 26898 raft_consensus.cc:1261] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47 [term 1 FOLLOWER]: Refusing update from remote peer 13010ce8672042709dd35b226269d8a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:08.128165 26972 consensus_queue.cc:1035] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12dd329c071d47358582fc9a836b4c5d" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 39095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:08.128314 26967 consensus_queue.cc:1035] T e9ab92907295404ea95aec71dbc2f9fd P 13010ce8672042709dd35b226269d8a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8820dd163a914008ab5826b445024e47" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:08.137939 26976 mvcc.cc:205] Tried to move back new op lower bound from 7298938585602183168 to 7298938585480736768. Current Snapshot: MvccSnapshot[applied={T|T < 7298938585602183168}]
I20260620 14:57:08.146055 26484 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:08.146278 26484 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:08.148895 26484 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b69c470953a402c90a42a5844d40624: Sending DeleteTablet for 3 replicas of tablet e9ab92907295404ea95aec71dbc2f9fd
I20260620 14:57:08.149808 26744 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ab92907295404ea95aec71dbc2f9fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:08 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:44480
I20260620 14:57:08.150357 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 26547
I20260620 14:57:08.150264 26878 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ab92907295404ea95aec71dbc2f9fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:08 UTC) from {username='kudu', principal='kudu/127.7.24.254@KRBTEST.COM'} at 127.0.0.1:46446
I20260620 14:57:08.151156 26986 tablet_replica.cc:330] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47: stopping tablet replica
I20260620 14:57:08.151679 26986 raft_consensus.cc:2229] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:57:08.152185 26986 raft_consensus.cc:2258] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:08.153323 26985 tablet_replica.cc:330] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d: stopping tablet replica
I20260620 14:57:08.153635 26985 raft_consensus.cc:2229] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:57:08.153945 26985 raft_consensus.cc:2258] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:08.154248 26986 ts_tablet_manager.cc:1905] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:08.155323 26985 ts_tablet_manager.cc:1905] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:08.157306 26986 ts_tablet_manager.cc:1918] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:57:08.157404 26986 log.cc:1196] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/e9ab92907295404ea95aec71dbc2f9fd
I20260620 14:57:08.157735 26986 ts_tablet_manager.cc:1939] T e9ab92907295404ea95aec71dbc2f9fd P 8820dd163a914008ab5826b445024e47: Deleting consensus metadata
I20260620 14:57:08.158152 26985 ts_tablet_manager.cc:1918] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:57:08.158246 26985 log.cc:1196] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/e9ab92907295404ea95aec71dbc2f9fd
W20260620 14:57:08.158672 26469 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:34207 (error 108)
I20260620 14:57:08.158766 26985 ts_tablet_manager.cc:1939] T e9ab92907295404ea95aec71dbc2f9fd P 12dd329c071d47358582fc9a836b4c5d: Deleting consensus metadata
I20260620 14:57:08.158929 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 26681
I20260620 14:57:08.159560 26472 catalog_manager.cc:4841] TS 8820dd163a914008ab5826b445024e47 (127.7.24.195:34911): tablet e9ab92907295404ea95aec71dbc2f9fd (table test-table [id=b5596f32213d405a8d52f31c8b3789de]) successfully deleted
I20260620 14:57:08.160953 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.160579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:34207 (local address 127.0.0.1:44498)
0620 14:57:08.160761 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:08.160847 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:34207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260620 14:57:08.161144 26469 catalog_manager.cc:4584] TS 13010ce8672042709dd35b226269d8a1 (127.7.24.193:34207): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9ab92907295404ea95aec71dbc2f9fd: Network error: Client connection negotiation failed: client connection to 127.7.24.193:34207: connect: Connection refused (error 111)
I20260620 14:57:08.165025 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 26815
I20260620 14:57:08.166252 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.165280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:39095 (local address 127.0.0.1:44490)
0620 14:57:08.166076 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:08.166144 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.194:39095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":694}
I20260620 14:57:08.170873 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 26453
2026-06-20T14:57:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2330 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-20T14:57:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:08Z Disabled control of system clock
I20260620 14:57:08.201803 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:38963
--webserver_interface=127.7.24.254
--webserver_port=0
--builtin_ntp_servers=127.7.24.212:42197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:38963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 14:57:08.311990 26991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:08.312258 26991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:08.312302 26991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:08.316207 26991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:08.316267 26991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:08.316282 26991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:08.316303 26991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:08.316319 26991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:08.321022 26991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:42197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:38963
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:38963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:08.322016 26991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:08.323360 26991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:08.332006 26997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:08.332031 26999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:08.332134 26991 server_base.cc:1034] running on GCE node
W20260620 14:57:08.332283 26996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:08.332861 26991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:08.333937 26991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:08.335103 26991 hybrid_clock.cc:648] HybridClock initialized: now 1781967428335083 us; error 32 us; skew 500 ppm
I20260620 14:57:08.336968 26991 webserver.cc:460] Webserver started at http://127.7.24.254:36195/ using document root <none> and password file <none>
I20260620 14:57:08.337626 26991 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:08.337692 26991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:08.337918 26991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:08.340101 26991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "b4af8ba3bc50445d830bae20b105191a"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "505066bf58f6f22a06ef2b4253167be5"
server_key_iv: "74ba500376d348dfb6eff43428d09b3f"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.340695 26991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "b4af8ba3bc50445d830bae20b105191a"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "505066bf58f6f22a06ef2b4253167be5"
server_key_iv: "74ba500376d348dfb6eff43428d09b3f"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.344521 26991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260620 14:57:08.347047 27005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:08.348363 26991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:08.348498 26991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "b4af8ba3bc50445d830bae20b105191a"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "505066bf58f6f22a06ef2b4253167be5"
server_key_iv: "74ba500376d348dfb6eff43428d09b3f"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.348608 26991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:08.364640 26991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:08.365350 26991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:08.365541 26991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:08.372718 26991 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:38963
I20260620 14:57:08.372726 27057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:38963 every 8 connection(s)
I20260620 14:57:08.373697 26991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:08.376869 27058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:08.377941 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 26991
I20260620 14:57:08.378072 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:08.378422 7267 external_mini_cluster.cc:1428] Setting key 7a7a4c9572dcd8002cc50168793c51cf
I20260620 14:57:08.383482 27058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a: Bootstrap starting.
I20260620 14:57:08.386332 27058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:08.387187 27058 log.cc:824] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:08.388536 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.380005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.7.24.254:38963)
0620 14:57:08.380620 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:08.380636 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:08.380673 (+ 37us) server_negotiation.cc:407] Connection header received
0620 14:57:08.381325 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:08.381343 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:08.381724 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:08.382011 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:08.382420 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.383434 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.384627 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.385209 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.385408 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:08.386508 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:08.386516 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:08.386535 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:08.386962 (+ 427us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:08.387020 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:08.387040 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:08.387118 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:08.387356 (+ 238us) server_negotiation.cc:299] Negotiation successful
0620 14:57:08.387679 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":128,"threads_started":1}
I20260620 14:57:08.389299 27058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a: No bootstrap required, opened a new log
I20260620 14:57:08.392956 27058 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4af8ba3bc50445d830bae20b105191a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38963 } }
I20260620 14:57:08.393208 27058 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:08.393290 27058 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4af8ba3bc50445d830bae20b105191a, State: Initialized, Role: FOLLOWER
I20260620 14:57:08.393708 27058 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4af8ba3bc50445d830bae20b105191a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38963 } }
I20260620 14:57:08.393870 27058 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:08.393956 27058 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:08.394074 27058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:08.395100 27058 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4af8ba3bc50445d830bae20b105191a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38963 } }
I20260620 14:57:08.395447 27058 leader_election.cc:302] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4af8ba3bc50445d830bae20b105191a; no voters:
I20260620 14:57:08.395794 27058 leader_election.cc:288] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:08.395944 27063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:08.396207 27063 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [term 1 LEADER]: Becoming Leader. State: Replica: b4af8ba3bc50445d830bae20b105191a, State: Running, Role: LEADER
I20260620 14:57:08.396595 27063 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4af8ba3bc50445d830bae20b105191a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38963 } }
I20260620 14:57:08.397076 27058 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:08.398241 27064 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4af8ba3bc50445d830bae20b105191a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4af8ba3bc50445d830bae20b105191a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38963 } } }
I20260620 14:57:08.398393 27064 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:08.398681 27065 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4af8ba3bc50445d830bae20b105191a. Latest consensus state: current_term: 1 leader_uuid: "b4af8ba3bc50445d830bae20b105191a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4af8ba3bc50445d830bae20b105191a" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38963 } } }
I20260620 14:57:08.398790 27065 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a [sys.catalog]: This master's current role is: LEADER
W20260620 14:57:08.402292 27078 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:57:08.402367 27078 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:57:08.402441 27072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:08.403280 27072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:08.408607 27072 catalog_manager.cc:1347] Generated new cluster ID: b50907c7cf3f42caa0b6eba968c0c266
I20260620 14:57:08.408685 27072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:08.415762 27072 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:08.416628 27072 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:08.422592 27072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a: Generated new TSK 0
I20260620 14:57:08.423224 27072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:08.429809 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.193:0
--local_ip_for_outbound_sockets=127.7.24.193
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:38963
--builtin_ntp_servers=127.7.24.212:42197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 14:57:08.539958 27082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:08.540205 27082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:08.540244 27082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:08.544363 27082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:08.544420 27082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:08.544476 27082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.193
I20260620 14:57:08.549201 27082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:42197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.24.193
--webserver_port=0
--tserver_master_addrs=127.7.24.254:38963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.193
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:08.550240 27082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:08.551594 27082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:08.561173 27090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:08.561309 27088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:08.561169 27087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:08.561481 27082 server_base.cc:1034] running on GCE node
I20260620 14:57:08.561930 27082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:08.562649 27082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:08.563851 27082 hybrid_clock.cc:648] HybridClock initialized: now 1781967428563840 us; error 28 us; skew 500 ppm
I20260620 14:57:08.572846 27082 webserver.cc:460] Webserver started at http://127.7.24.193:35989/ using document root <none> and password file <none>
I20260620 14:57:08.573467 27082 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:08.573565 27082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:08.573800 27082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:08.575994 27082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/data/instance:
uuid: "513e01ab6f0043d48d557dd682cc9e8e"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "0b0aab8120eda3207e9769b48dc200ec"
server_key_iv: "46818f7aa67116d53e67c7f5a54157ab"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.576562 27082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance:
uuid: "513e01ab6f0043d48d557dd682cc9e8e"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "0b0aab8120eda3207e9769b48dc200ec"
server_key_iv: "46818f7aa67116d53e67c7f5a54157ab"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.580161 27082 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 14:57:08.582616 27096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:08.583786 27082 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:08.583920 27082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/wal
uuid: "513e01ab6f0043d48d557dd682cc9e8e"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "0b0aab8120eda3207e9769b48dc200ec"
server_key_iv: "46818f7aa67116d53e67c7f5a54157ab"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.584033 27082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:08.601593 27082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:08.602281 27082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:08.602551 27082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:08.603143 27082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:08.604094 27082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:08.604166 27082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:08.604255 27082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:08.604314 27082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:08.613472 27082 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.193:43819
I20260620 14:57:08.613497 27209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.193:43819 every 8 connection(s)
I20260620 14:57:08.614390 27082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/data/info.pb
I20260620 14:57:08.616276 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 27082
I20260620 14:57:08.616400 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-0/wal/instance
I20260620 14:57:08.616705 7267 external_mini_cluster.cc:1428] Setting key 212081ab0ac7890a54bd439ea7e82ac6
I20260620 14:57:08.618762 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.194:0
--local_ip_for_outbound_sockets=127.7.24.194
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:38963
--builtin_ntp_servers=127.7.24.212:42197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 14:57:08.624117 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.616355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:56919 (local address 127.7.24.254:38963)
0620 14:57:08.616509 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:08.616513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:08.617917 (+ 1404us) server_negotiation.cc:407] Connection header received
0620 14:57:08.619452 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:08.619456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:08.619516 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:08.619623 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:08.620487 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.621019 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.622139 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.622391 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.622521 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:08.623631 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:08.623634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:08.623636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:08.623647 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:08.623666 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:08.623673 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:08.623747 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:08.623906 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 14:57:08.623974 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 14:57:08.624945 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.616833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38963 (local address 127.7.24.193:56919)
0620 14:57:08.617772 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:08.617812 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:08.619236 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:08.619794 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:08.619801 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:08.619883 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:08.620340 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:08.620352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.621145 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.621149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:08.622011 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:08.622020 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.622564 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.622568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:08.622725 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:08.623356 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:57:08.623358 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:08.623395 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:57:08.623456 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:08.623753 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:08.623764 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:08.623771 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 14:57:08.623905 (+ 134us) client_negotiation.cc:240] Negotiation successful
0620 14:57:08.624164 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":135,"threads_started":1}
I20260620 14:57:08.626122 27210 heartbeater.cc:347] Connected to a master server at 127.7.24.254:38963
I20260620 14:57:08.626386 27210 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:08.627060 27210 heartbeater.cc:511] Master 127.7.24.254:38963 requested a full tablet report, sending...
I20260620 14:57:08.628661 27022 ts_manager.cc:194] Registered new tserver with Master: 513e01ab6f0043d48d557dd682cc9e8e (127.7.24.193:43819)
I20260620 14:57:08.629882 27022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.193:56919
W20260620 14:57:08.729419 27213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:08.729666 27213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:08.729704 27213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:08.733747 27213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:08.733801 27213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:08.733886 27213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.194
I20260620 14:57:08.738713 27213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:42197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.24.194
--webserver_port=0
--tserver_master_addrs=127.7.24.254:38963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.194
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:08.739770 27213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:08.741173 27213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:08.750828 27218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:08.750976 27213 server_base.cc:1034] running on GCE node
W20260620 14:57:08.750802 27221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:08.750813 27219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:08.751417 27213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:08.752104 27213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:08.753328 27213 hybrid_clock.cc:648] HybridClock initialized: now 1781967428753308 us; error 32 us; skew 500 ppm
I20260620 14:57:08.755275 27213 webserver.cc:460] Webserver started at http://127.7.24.194:44699/ using document root <none> and password file <none>
I20260620 14:57:08.755889 27213 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:08.755952 27213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:08.756204 27213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:08.758560 27213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/data/instance:
uuid: "94e2a6df008a49029eb32b91e29e33ab"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "cfd980d94d76ac53c3e55bb6c44f36a3"
server_key_iv: "df8099c7e5f12652544d8e09891a45f4"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.759131 27213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance:
uuid: "94e2a6df008a49029eb32b91e29e33ab"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "cfd980d94d76ac53c3e55bb6c44f36a3"
server_key_iv: "df8099c7e5f12652544d8e09891a45f4"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.763003 27213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 14:57:08.765452 27227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:08.766690 27213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:08.766820 27213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/wal
uuid: "94e2a6df008a49029eb32b91e29e33ab"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "cfd980d94d76ac53c3e55bb6c44f36a3"
server_key_iv: "df8099c7e5f12652544d8e09891a45f4"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.766945 27213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:08.782606 27213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:08.783354 27213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:08.783565 27213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:08.784148 27213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:08.785094 27213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:08.785174 27213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:08.785254 27213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:08.785300 27213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:08.794497 27213 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.194:35261
I20260620 14:57:08.794566 27340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.194:35261 every 8 connection(s)
I20260620 14:57:08.795465 27213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/data/info.pb
I20260620 14:57:08.804980 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 27213
I20260620 14:57:08.805140 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/wal/instance
I20260620 14:57:08.805429 7267 external_mini_cluster.cc:1428] Setting key e5f3aaf3675c8679e9cf719cee651c89
I20260620 14:57:08.805518 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.797426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.194:38889 (local address 127.7.24.254:38963)
0620 14:57:08.797579 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:08.797582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:08.799073 (+ 1491us) server_negotiation.cc:407] Connection header received
0620 14:57:08.800913 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:08.800916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:08.800965 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:08.801049 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:08.801890 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.802439 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.803572 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.803834 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.803927 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:08.804946 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:08.804950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:08.804953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:08.804966 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:08.804992 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:08.804999 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:08.805075 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:08.805244 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 14:57:08.805360 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:57:08.806388 27343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.797775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38963 (local address 127.7.24.194:38889)
0620 14:57:08.798915 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:08.798953 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:08.800708 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:08.801196 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:08.801203 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:08.801283 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:08.801739 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:08.801750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.802548 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.802550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:08.803438 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:08.803447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.803973 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.803976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:08.804109 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:08.804682 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:57:08.804683 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:08.804721 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:57:08.804779 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:08.805089 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:08.805100 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:08.805105 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 14:57:08.805227 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 14:57:08.805492 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":151,"threads_started":1}
I20260620 14:57:08.807654 27341 heartbeater.cc:347] Connected to a master server at 127.7.24.254:38963
I20260620 14:57:08.807682 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.24.195:0
--local_ip_for_outbound_sockets=127.7.24.195
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:38963
--builtin_ntp_servers=127.7.24.212:42197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 14:57:08.807897 27341 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:08.808409 27341 heartbeater.cc:511] Master 127.7.24.254:38963 requested a full tablet report, sending...
I20260620 14:57:08.809453 27022 ts_manager.cc:194] Registered new tserver with Master: 94e2a6df008a49029eb32b91e29e33ab (127.7.24.194:35261)
I20260620 14:57:08.810248 27022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.194:38889
W20260620 14:57:08.919137 27344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:08.919409 27344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:08.919448 27344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:08.923628 27344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:08.923684 27344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:08.923739 27344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.24.195
I20260620 14:57:08.928615 27344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:42197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.24.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.24.195
--webserver_port=0
--tserver_master_addrs=127.7.24.254:38963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.24.195
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:08.929708 27344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:08.931133 27344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:08.941162 27349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:08.941243 27350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:08.941272 27352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:08.941411 27344 server_base.cc:1034] running on GCE node
I20260620 14:57:08.941913 27344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:08.942651 27344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:08.943838 27344 hybrid_clock.cc:648] HybridClock initialized: now 1781967428943803 us; error 47 us; skew 500 ppm
I20260620 14:57:08.945676 27344 webserver.cc:460] Webserver started at http://127.7.24.195:45965/ using document root <none> and password file <none>
I20260620 14:57:08.946259 27344 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:08.946323 27344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:08.946595 27344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:08.948614 27344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/data/instance:
uuid: "d97950425b3c4845957bdf29dbd8a7fd"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "de1f3c0cbabc6fb6d1f31a57eff8ff9c"
server_key_iv: "21346eb86cde5d162688799716f25a01"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.949193 27344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance:
uuid: "d97950425b3c4845957bdf29dbd8a7fd"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "de1f3c0cbabc6fb6d1f31a57eff8ff9c"
server_key_iv: "21346eb86cde5d162688799716f25a01"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.952976 27344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260620 14:57:08.955561 27358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:08.956807 27344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:08.956964 27344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/wal
uuid: "d97950425b3c4845957bdf29dbd8a7fd"
format_stamp: "Formatted at 2026-06-20 14:57:08 on dist-test-slave-sms9"
server_key: "de1f3c0cbabc6fb6d1f31a57eff8ff9c"
server_key_iv: "21346eb86cde5d162688799716f25a01"
server_key_version: "encryptionkey@0"
I20260620 14:57:08.957075 27344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:08.977366 27344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:08.978085 27344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:08.978281 27344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:08.978953 27344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 14:57:08.979918 27344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 14:57:08.979964 27344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:08.980005 27344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 14:57:08.980057 27344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:08.989624 27344 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.195:45825
I20260620 14:57:08.989775 27471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.195:45825 every 8 connection(s)
I20260620 14:57:08.990638 27344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/data/info.pb
I20260620 14:57:08.994753 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 27344
I20260620 14:57:08.994885 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/wal/instance
I20260620 14:57:08.995150 7267 external_mini_cluster.cc:1428] Setting key f43516269096459cfbd9307dc5d2d5b6
I20260620 14:57:09.000629 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.992492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.195:56593 (local address 127.7.24.254:38963)
0620 14:57:08.992653 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:08.992657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:08.993625 (+ 968us) server_negotiation.cc:407] Connection header received
0620 14:57:08.995135 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:08.995139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:08.995214 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:08.995320 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:08.996600 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.997370 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.998691 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.998953 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.999035 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.000179 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:09.000182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:09.000184 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:09.000196 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:09.000215 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:09.000223 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:09.000282 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:09.000471 (+ 189us) server_negotiation.cc:299] Negotiation successful
0620 14:57:09.000521 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 14:57:09.001544 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:08.992846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38963 (local address 127.7.24.195:56593)
0620 14:57:08.993472 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:08.993511 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:08.994897 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:08.995528 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:08.995535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:08.995617 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:08.996402 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:08.996429 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.997528 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.997532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:08.998582 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:08.998592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:08.999156 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:08.999164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:08.999317 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:08.999929 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 14:57:08.999931 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:08.999970 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 14:57:09.000032 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:09.000325 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:09.000333 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:09.000338 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 14:57:09.000469 (+ 131us) client_negotiation.cc:240] Negotiation successful
0620 14:57:09.000723 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":113,"threads_started":1}
I20260620 14:57:09.002842 27472 heartbeater.cc:347] Connected to a master server at 127.7.24.254:38963
I20260620 14:57:09.003086 27472 heartbeater.cc:464] Registering TS with master...
I20260620 14:57:09.003607 27472 heartbeater.cc:511] Master 127.7.24.254:38963 requested a full tablet report, sending...
I20260620 14:57:09.004633 27022 ts_manager.cc:194] Registered new tserver with Master: d97950425b3c4845957bdf29dbd8a7fd (127.7.24.195:45825)
I20260620 14:57:09.005360 27022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.24.195:56593
I20260620 14:57:09.009213 7267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 14:57:09.015048 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.011198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41896 (local address 127.7.24.254:38963)
0620 14:57:09.011341 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:09.011344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:09.011501 (+ 157us) server_negotiation.cc:407] Connection header received
0620 14:57:09.011595 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:09.011597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:09.011634 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:09.011705 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:09.011967 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.012466 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.013357 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.013582 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.013664 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.014645 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:09.014648 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:09.014650 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:09.014661 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:09.014679 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:09.014686 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:09.014745 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:09.014886 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 14:57:09.014941 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 14:57:09.017812 27022 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 14:57:09.020884 27022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 14:57:09.035737 27484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.029507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:45825 (local address 127.0.0.1:58052)
0620 14:57:09.029928 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:09.029972 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:09.030093 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:09.031079 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:09.031086 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:09.031107 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:09.031251 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:09.031258 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.032655 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.032658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:09.034254 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:09.034265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.035193 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.035197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:09.035315 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.035352 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 14:57:09.035451 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 14:57:09.035567 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":171,"thread_start_us":75,"threads_started":1}
I20260620 14:57:09.035750 27485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.029567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58052 (local address 127.7.24.195:45825)
0620 14:57:09.030389 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:09.030401 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:09.030426 (+ 25us) server_negotiation.cc:407] Connection header received
0620 14:57:09.030526 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:09.030533 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:09.030759 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:09.030907 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:09.031380 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.032543 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.034543 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.035068 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.035194 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.035464 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:09.035512 (+ 48us) server_negotiation.cc:299] Negotiation successful
0620 14:57:09.035602 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":120,"threads_started":1}
I20260620 14:57:09.036147 27486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.030211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:35261 (local address 127.0.0.1:45876)
0620 14:57:09.031123 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:09.031136 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:09.031215 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:09.031968 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:09.031971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:09.031991 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:09.032082 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:09.032088 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.033232 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.033236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:09.034269 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:09.034279 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.035562 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.035566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:09.035797 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.035857 (+ 60us) client_negotiation.cc:769] Sending connection context
0620 14:57:09.035932 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 14:57:09.036012 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":139,"thread_start_us":78,"threads_started":1}
I20260620 14:57:09.036245 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.030451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33662 (local address 127.7.24.193:43819)
0620 14:57:09.030879 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:09.030889 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:09.030909 (+ 20us) server_negotiation.cc:407] Connection header received
0620 14:57:09.030975 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:09.030978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:09.031138 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:09.031291 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:09.031539 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.032438 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.034388 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.035218 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.035324 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.035495 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:09.035996 (+ 501us) server_negotiation.cc:299] Negotiation successful
0620 14:57:09.036089 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":105,"threads_started":1}
I20260620 14:57:09.036639 27488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.030633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45876 (local address 127.7.24.194:35261)
0620 14:57:09.031557 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:09.031567 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:09.031588 (+ 21us) server_negotiation.cc:407] Connection header received
0620 14:57:09.031656 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:09.031661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:09.031821 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:09.031962 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:09.032203 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.033067 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.034426 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.035224 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.035324 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.035495 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:09.036321 (+ 826us) server_negotiation.cc:299] Negotiation successful
0620 14:57:09.036485 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":148,"threads_started":1}
I20260620 14:57:09.036705 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.029553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:43819 (local address 127.0.0.1:33662)
0620 14:57:09.030515 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:09.030531 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:09.030620 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:09.031292 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:09.031295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:09.031321 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:09.031421 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:09.031425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.032542 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.032548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:09.034253 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:09.034265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.035562 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.035566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:09.035795 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.035855 (+ 60us) client_negotiation.cc:769] Sending connection context
0620 14:57:09.036501 (+ 646us) client_negotiation.cc:240] Negotiation successful
0620 14:57:09.036577 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":261,"thread_start_us":126,"threads_started":1}
I20260620 14:57:09.039449 27144 tablet_service.cc:1467] Processing CreateTablet for tablet 36df205d90e24662a2f490c2ae68e906 (DEFAULT_TABLE table=test-table [id=7c6f25d0bdd845a0ba09c23fb91a361a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:09.039404 27406 tablet_service.cc:1467] Processing CreateTablet for tablet 36df205d90e24662a2f490c2ae68e906 (DEFAULT_TABLE table=test-table [id=7c6f25d0bdd845a0ba09c23fb91a361a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:09.039404 27275 tablet_service.cc:1467] Processing CreateTablet for tablet 36df205d90e24662a2f490c2ae68e906 (DEFAULT_TABLE table=test-table [id=7c6f25d0bdd845a0ba09c23fb91a361a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 14:57:09.040400 27144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36df205d90e24662a2f490c2ae68e906. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:09.040421 27275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36df205d90e24662a2f490c2ae68e906. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:09.040421 27406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36df205d90e24662a2f490c2ae68e906. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:09.047276 27489 tablet_bootstrap.cc:492] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e: Bootstrap starting.
I20260620 14:57:09.047940 27491 tablet_bootstrap.cc:492] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: Bootstrap starting.
I20260620 14:57:09.049259 27489 tablet_bootstrap.cc:654] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:09.049559 27491 tablet_bootstrap.cc:654] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:09.050024 27489 log.cc:824] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:09.050240 27491 log.cc:824] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:09.050791 27490 tablet_bootstrap.cc:492] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: Bootstrap starting.
I20260620 14:57:09.052183 27489 tablet_bootstrap.cc:492] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e: No bootstrap required, opened a new log
I20260620 14:57:09.052183 27491 tablet_bootstrap.cc:492] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: No bootstrap required, opened a new log
I20260620 14:57:09.052393 27489 ts_tablet_manager.cc:1397] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260620 14:57:09.052407 27491 ts_tablet_manager.cc:1397] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 14:57:09.052754 27490 tablet_bootstrap.cc:654] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:09.053439 27490 log.cc:824] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:09.055614 27490 tablet_bootstrap.cc:492] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: No bootstrap required, opened a new log
I20260620 14:57:09.055823 27490 ts_tablet_manager.cc:1397] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 14:57:09.056959 27491 raft_consensus.cc:348] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97950425b3c4845957bdf29dbd8a7fd" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45825 } } peers { permanent_uuid: "513e01ab6f0043d48d557dd682cc9e8e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43819 } } peers { permanent_uuid: "94e2a6df008a49029eb32b91e29e33ab" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 35261 } }
I20260620 14:57:09.056958 27489 raft_consensus.cc:348] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97950425b3c4845957bdf29dbd8a7fd" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45825 } } peers { permanent_uuid: "513e01ab6f0043d48d557dd682cc9e8e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43819 } } peers { permanent_uuid: "94e2a6df008a49029eb32b91e29e33ab" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 35261 } }
I20260620 14:57:09.057237 27491 raft_consensus.cc:374] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:09.057305 27491 raft_consensus.cc:729] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97950425b3c4845957bdf29dbd8a7fd, State: Initialized, Role: FOLLOWER
I20260620 14:57:09.057318 27489 raft_consensus.cc:374] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:09.057369 27489 raft_consensus.cc:729] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513e01ab6f0043d48d557dd682cc9e8e, State: Initialized, Role: FOLLOWER
I20260620 14:57:09.057785 27491 consensus_queue.cc:260] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97950425b3c4845957bdf29dbd8a7fd" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45825 } } peers { permanent_uuid: "513e01ab6f0043d48d557dd682cc9e8e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43819 } } peers { permanent_uuid: "94e2a6df008a49029eb32b91e29e33ab" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 35261 } }
I20260620 14:57:09.057780 27489 consensus_queue.cc:260] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97950425b3c4845957bdf29dbd8a7fd" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45825 } } peers { permanent_uuid: "513e01ab6f0043d48d557dd682cc9e8e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43819 } } peers { permanent_uuid: "94e2a6df008a49029eb32b91e29e33ab" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 35261 } }
I20260620 14:57:09.058508 27210 heartbeater.cc:503] Master 127.7.24.254:38963 was elected leader, sending a full tablet report...
I20260620 14:57:09.058764 27472 heartbeater.cc:503] Master 127.7.24.254:38963 was elected leader, sending a full tablet report...
I20260620 14:57:09.058841 27491 ts_tablet_manager.cc:1428] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 14:57:09.058841 27489 ts_tablet_manager.cc:1428] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260620 14:57:09.061370 27490 raft_consensus.cc:348] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97950425b3c4845957bdf29dbd8a7fd" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45825 } } peers { permanent_uuid: "513e01ab6f0043d48d557dd682cc9e8e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43819 } } peers { permanent_uuid: "94e2a6df008a49029eb32b91e29e33ab" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 35261 } }
I20260620 14:57:09.061599 27490 raft_consensus.cc:374] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:09.061676 27490 raft_consensus.cc:729] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94e2a6df008a49029eb32b91e29e33ab, State: Initialized, Role: FOLLOWER
I20260620 14:57:09.062129 27490 consensus_queue.cc:260] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97950425b3c4845957bdf29dbd8a7fd" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45825 } } peers { permanent_uuid: "513e01ab6f0043d48d557dd682cc9e8e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43819 } } peers { permanent_uuid: "94e2a6df008a49029eb32b91e29e33ab" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 35261 } }
I20260620 14:57:09.062795 27341 heartbeater.cc:503] Master 127.7.24.254:38963 was elected leader, sending a full tablet report...
I20260620 14:57:09.064006 27490 ts_tablet_manager.cc:1428] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20260620 14:57:09.115615 27211 tablet.cc:2369] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 14:57:09.241685 27473 tablet.cc:2369] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:57:09.287678 27495 raft_consensus.cc:482] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 14:57:09.287962 27495 raft_consensus.cc:504] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97950425b3c4845957bdf29dbd8a7fd" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45825 } } peers { permanent_uuid: "513e01ab6f0043d48d557dd682cc9e8e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43819 } } peers { permanent_uuid: "94e2a6df008a49029eb32b91e29e33ab" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 35261 } }
I20260620 14:57:09.289247 27495 leader_election.cc:288] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d97950425b3c4845957bdf29dbd8a7fd (127.7.24.195:45825), 94e2a6df008a49029eb32b91e29e33ab (127.7.24.194:35261)
I20260620 14:57:09.293375 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.289588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.195:45825 (local address 127.7.24.193:44771)
0620 14:57:09.289964 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:09.289977 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:09.290124 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:09.290512 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:09.290515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:09.290534 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:09.290683 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:09.290689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.291435 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.291441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:09.292242 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:09.292251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.292902 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.292906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:09.293002 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.293024 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 14:57:09.293122 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 14:57:09.293224 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":14,"thread_start_us":145,"threads_started":1}
I20260620 14:57:09.293354 27485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.289705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:44771 (local address 127.7.24.195:45825)
0620 14:57:09.289873 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:09.289878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:09.290067 (+ 189us) server_negotiation.cc:407] Connection header received
0620 14:57:09.290252 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:09.290257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:09.290366 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:09.290501 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:09.290797 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.291323 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.292375 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.292767 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.292902 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.292954 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:09.293137 (+ 183us) server_negotiation.cc:299] Negotiation successful
0620 14:57:09.293208 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 14:57:09.293586 27488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.290177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.24.193:51971 (local address 127.7.24.194:35261)
0620 14:57:09.290346 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:09.290353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:09.290491 (+ 138us) server_negotiation.cc:407] Connection header received
0620 14:57:09.290623 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:09.290626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:09.290679 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:09.290772 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 14:57:09.291049 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.291540 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.292561 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.293007 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.293132 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.293179 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:09.293407 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 14:57:09.293476 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260620 14:57:09.293612 27499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.290053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.194:35261 (local address 127.7.24.193:51971)
0620 14:57:09.290371 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:09.290383 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:09.290528 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:09.290784 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:09.290786 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:09.290801 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 14:57:09.290929 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:09.290934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.291657 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.291662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:09.292432 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:09.292441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.293145 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.293148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:09.293286 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.293303 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 14:57:09.293392 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 14:57:09.293457 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":23,"thread_start_us":65,"threads_started":1}
I20260620 14:57:09.293999 27426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36df205d90e24662a2f490c2ae68e906" candidate_uuid: "513e01ab6f0043d48d557dd682cc9e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97950425b3c4845957bdf29dbd8a7fd" is_pre_election: true
I20260620 14:57:09.294288 27295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36df205d90e24662a2f490c2ae68e906" candidate_uuid: "513e01ab6f0043d48d557dd682cc9e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e2a6df008a49029eb32b91e29e33ab" is_pre_election: true
I20260620 14:57:09.294313 27426 raft_consensus.cc:2454] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513e01ab6f0043d48d557dd682cc9e8e in term 0.
I20260620 14:57:09.294585 27295 raft_consensus.cc:2454] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513e01ab6f0043d48d557dd682cc9e8e in term 0.
I20260620 14:57:09.294859 27099 leader_election.cc:302] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 513e01ab6f0043d48d557dd682cc9e8e, d97950425b3c4845957bdf29dbd8a7fd; no voters:
I20260620 14:57:09.295132 27495 raft_consensus.cc:2781] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 14:57:09.295236 27495 raft_consensus.cc:482] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 14:57:09.295282 27495 raft_consensus.cc:3037] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:09.296469 27495 raft_consensus.cc:504] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97950425b3c4845957bdf29dbd8a7fd" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45825 } } peers { permanent_uuid: "513e01ab6f0043d48d557dd682cc9e8e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43819 } } peers { permanent_uuid: "94e2a6df008a49029eb32b91e29e33ab" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 35261 } }
W20260620 14:57:09.296693 27342 tablet.cc:2369] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:57:09.296855 27495 leader_election.cc:288] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [CANDIDATE]: Term 1 election: Requested vote from peers d97950425b3c4845957bdf29dbd8a7fd (127.7.24.195:45825), 94e2a6df008a49029eb32b91e29e33ab (127.7.24.194:35261)
I20260620 14:57:09.297550 27295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36df205d90e24662a2f490c2ae68e906" candidate_uuid: "513e01ab6f0043d48d557dd682cc9e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e2a6df008a49029eb32b91e29e33ab"
I20260620 14:57:09.297495 27426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36df205d90e24662a2f490c2ae68e906" candidate_uuid: "513e01ab6f0043d48d557dd682cc9e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d97950425b3c4845957bdf29dbd8a7fd"
I20260620 14:57:09.297689 27426 raft_consensus.cc:3037] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:09.297679 27295 raft_consensus.cc:3037] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:09.299288 27295 raft_consensus.cc:2454] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513e01ab6f0043d48d557dd682cc9e8e in term 1.
I20260620 14:57:09.299306 27426 raft_consensus.cc:2454] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513e01ab6f0043d48d557dd682cc9e8e in term 1.
I20260620 14:57:09.299636 27099 leader_election.cc:302] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 513e01ab6f0043d48d557dd682cc9e8e, d97950425b3c4845957bdf29dbd8a7fd; no voters:
I20260620 14:57:09.299817 27495 raft_consensus.cc:2781] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:09.300082 27495 raft_consensus.cc:686] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [term 1 LEADER]: Becoming Leader. State: Replica: 513e01ab6f0043d48d557dd682cc9e8e, State: Running, Role: LEADER
I20260620 14:57:09.300479 27495 consensus_queue.cc:237] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97950425b3c4845957bdf29dbd8a7fd" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45825 } } peers { permanent_uuid: "513e01ab6f0043d48d557dd682cc9e8e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43819 } } peers { permanent_uuid: "94e2a6df008a49029eb32b91e29e33ab" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 35261 } }
I20260620 14:57:09.304605 27022 catalog_manager.cc:5495] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 513e01ab6f0043d48d557dd682cc9e8e (127.7.24.193). New cstate: current_term: 1 leader_uuid: "513e01ab6f0043d48d557dd682cc9e8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d97950425b3c4845957bdf29dbd8a7fd" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "513e01ab6f0043d48d557dd682cc9e8e" member_type: VOTER last_known_addr { host: "127.7.24.193" port: 43819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94e2a6df008a49029eb32b91e29e33ab" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 35261 } health_report { overall_health: UNKNOWN } } }
I20260620 14:57:09.329560 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.325896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33676 (local address 127.7.24.193:43819)
0620 14:57:09.326039 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:09.326042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:09.326055 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:57:09.326131 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:09.326133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:09.326194 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:09.326300 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:09.326604 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.327111 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.327971 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:09.328190 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:09.328336 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:09.328625 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:09.329064 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:09.329157 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:09.329292 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 14:57:09.329344 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 14:57:09.335561 27426 raft_consensus.cc:1261] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd [term 1 FOLLOWER]: Refusing update from remote peer 513e01ab6f0043d48d557dd682cc9e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:09.335561 27295 raft_consensus.cc:1261] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab [term 1 FOLLOWER]: Refusing update from remote peer 513e01ab6f0043d48d557dd682cc9e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:09.336266 27495 consensus_queue.cc:1035] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "94e2a6df008a49029eb32b91e29e33ab" member_type: VOTER last_known_addr { host: "127.7.24.194" port: 35261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:09.336534 27500 consensus_queue.cc:1035] T 36df205d90e24662a2f490c2ae68e906 P 513e01ab6f0043d48d557dd682cc9e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d97950425b3c4845957bdf29dbd8a7fd" member_type: VOTER last_known_addr { host: "127.7.24.195" port: 45825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:09.342514 27503 mvcc.cc:205] Tried to move back new op lower bound from 7298938590550761472 to 7298938590418362368. Current Snapshot: MvccSnapshot[applied={T|T < 7298938590550761472}]
I20260620 14:57:09.343976 27504 mvcc.cc:205] Tried to move back new op lower bound from 7298938590550761472 to 7298938590418362368. Current Snapshot: MvccSnapshot[applied={T|T < 7298938590550761472}]
I20260620 14:57:09.346269 27505 mvcc.cc:205] Tried to move back new op lower bound from 7298938590550761472 to 7298938590418362368. Current Snapshot: MvccSnapshot[applied={T|T < 7298938590550761472}]
I20260620 14:57:09.351774 27022 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:09.351941 27022 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 14:57:09.354228 27022 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4af8ba3bc50445d830bae20b105191a: Sending DeleteTablet for 3 replicas of tablet 36df205d90e24662a2f490c2ae68e906
I20260620 14:57:09.355057 27144 tablet_service.cc:1514] Processing DeleteTablet for tablet 36df205d90e24662a2f490c2ae68e906 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:09 UTC) from {username='slave'} at 127.0.0.1:33662
I20260620 14:57:09.355310 27406 tablet_service.cc:1514] Processing DeleteTablet for tablet 36df205d90e24662a2f490c2ae68e906 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:09 UTC) from {username='slave'} at 127.0.0.1:58052
I20260620 14:57:09.355625 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 27082
I20260620 14:57:09.355764 27275 tablet_service.cc:1514] Processing DeleteTablet for tablet 36df205d90e24662a2f490c2ae68e906 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 14:57:09 UTC) from {username='slave'} at 127.0.0.1:45876
I20260620 14:57:09.357445 27514 tablet_replica.cc:330] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: stopping tablet replica
I20260620 14:57:09.357473 27513 tablet_replica.cc:330] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: stopping tablet replica
I20260620 14:57:09.357764 27514 raft_consensus.cc:2229] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:57:09.357764 27513 raft_consensus.cc:2229] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 14:57:09.358083 27513 raft_consensus.cc:2258] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:09.358083 27514 raft_consensus.cc:2258] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 14:57:09.359592 27513 ts_tablet_manager.cc:1905] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:09.359606 27514 ts_tablet_manager.cc:1905] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 14:57:09.362169 27514 ts_tablet_manager.cc:1918] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:57:09.362285 27514 log.cc:1196] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-1/wal/wals/36df205d90e24662a2f490c2ae68e906
I20260620 14:57:09.362619 27514 ts_tablet_manager.cc:1939] T 36df205d90e24662a2f490c2ae68e906 P 94e2a6df008a49029eb32b91e29e33ab: Deleting consensus metadata
I20260620 14:57:09.363037 27513 ts_tablet_manager.cc:1918] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 14:57:09.363145 27513 log.cc:1196] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: Deleting WAL directory at /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781967176660325-7267-0/minicluster-data/ts-2/wal/wals/36df205d90e24662a2f490c2ae68e906
I20260620 14:57:09.363469 27513 ts_tablet_manager.cc:1939] T 36df205d90e24662a2f490c2ae68e906 P d97950425b3c4845957bdf29dbd8a7fd: Deleting consensus metadata
I20260620 14:57:09.363930 27008 catalog_manager.cc:4841] TS 94e2a6df008a49029eb32b91e29e33ab (127.7.24.194:35261): tablet 36df205d90e24662a2f490c2ae68e906 (table test-table [id=7c6f25d0bdd845a0ba09c23fb91a361a]) successfully deleted
W20260620 14:57:09.364434 27007 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.24.193:43819 (error 108)
I20260620 14:57:09.364540 27008 catalog_manager.cc:4841] TS d97950425b3c4845957bdf29dbd8a7fd (127.7.24.195:45825): tablet 36df205d90e24662a2f490c2ae68e906 (table test-table [id=7c6f25d0bdd845a0ba09c23fb91a361a]) successfully deleted
I20260620 14:57:09.364931 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 27213
I20260620 14:57:09.366040 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:09.365686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.193:43819 (local address 127.0.0.1:33686)
0620 14:57:09.365846 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:09.365934 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.193:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260620 14:57:09.366250 27007 catalog_manager.cc:4584] TS 513e01ab6f0043d48d557dd682cc9e8e (127.7.24.193:43819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 36df205d90e24662a2f490c2ae68e906: Network error: Client connection negotiation failed: client connection to 127.7.24.193:43819: connect: Connection refused (error 111)
I20260620 14:57:09.370836 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 27344
I20260620 14:57:09.376421 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 26991
2026-06-20T14:57:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1204 ms)
[----------] 6 tests from EncryptionPolicyTest (10986 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:09 dist-test-slave-sms9 krb5kdc[27518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:09 dist-test-slave-sms9 krb5kdc[27518](info): set up 2 sockets
Jun 20 14:57:09 dist-test-slave-sms9 krb5kdc[27518](info): commencing operation
krb5kdc: starting...
W20260620 14:57:10.422605 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:10 dist-test-slave-sms9 krb5kdc[27518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:10.545363 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39907
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:10.656482 27531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:10.656805 27531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:10.656883 27531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:10.660959 27531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:10.661064 27531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:10.661106 27531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:10.661163 27531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:10.661203 27531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:10.666644 27531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39907
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:10.667765 27531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:10.668779 27531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:10.677553 27537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:10.677553 27539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:10.677557 27536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:10.677596 27531 server_base.cc:1034] running on GCE node
I20260620 14:57:10.678442 27531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:10.679629 27531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:10.680821 27531 hybrid_clock.cc:648] HybridClock initialized: now 1781967430680793 us; error 47 us; skew 500 ppm
Jun 20 14:57:10 dist-test-slave-sms9 krb5kdc[27518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967430, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:10.683874 27531 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:10.684902 27531 webserver.cc:460] Webserver started at http://127.0.0.1:44189/ using document root <none> and password file <none>
I20260620 14:57:10.685539 27531 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:10.685642 27531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:10.685904 27531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:10.688335 27531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "5a577685fbf94d6ebf7e2f3d70a85a9c"
format_stamp: "Formatted at 2026-06-20 14:57:10 on dist-test-slave-sms9"
server_key: "eb12aa808c3ff130b0e79836367c9c6d"
server_key_iv: "c2f55b6a4c0e814f0f4448e3c1486ea5"
server_key_version: "encryptionkey@0"
I20260620 14:57:10.689010 27531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "5a577685fbf94d6ebf7e2f3d70a85a9c"
format_stamp: "Formatted at 2026-06-20 14:57:10 on dist-test-slave-sms9"
server_key: "eb12aa808c3ff130b0e79836367c9c6d"
server_key_iv: "c2f55b6a4c0e814f0f4448e3c1486ea5"
server_key_version: "encryptionkey@0"
I20260620 14:57:10.693787 27531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:57:10.696511 27546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:10.697919 27531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 14:57:10.698071 27531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "5a577685fbf94d6ebf7e2f3d70a85a9c"
format_stamp: "Formatted at 2026-06-20 14:57:10 on dist-test-slave-sms9"
server_key: "eb12aa808c3ff130b0e79836367c9c6d"
server_key_iv: "c2f55b6a4c0e814f0f4448e3c1486ea5"
server_key_version: "encryptionkey@0"
I20260620 14:57:10.698197 27531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:10.719257 27531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:10.723048 27531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:10.723294 27531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:10.731091 27531 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39907
I20260620 14:57:10.731122 27598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39907 every 8 connection(s)
I20260620 14:57:10.732183 27531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:10.735137 27599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:10.740682 27599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c: Bootstrap starting.
I20260620 14:57:10.742116 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 27531
I20260620 14:57:10.742249 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:10.742578 7267 external_mini_cluster.cc:1428] Setting key c13880aaa615db1a9acdb21c1c56b647
I20260620 14:57:10.743637 27599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:10.744501 27599 log.cc:824] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:10.746690 27599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c: No bootstrap required, opened a new log
I20260620 14:57:10.751067 27599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a577685fbf94d6ebf7e2f3d70a85a9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39907 } }
I20260620 14:57:10.751322 27599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:10.751411 27599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a577685fbf94d6ebf7e2f3d70a85a9c, State: Initialized, Role: FOLLOWER
I20260620 14:57:10.751920 27599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a577685fbf94d6ebf7e2f3d70a85a9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39907 } }
I20260620 14:57:10.752094 27599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:10.752159 27599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:10.752260 27599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [term 0 FOLLOWER]: Advancing to term 1
Jun 20 14:57:10 dist-test-slave-sms9 krb5kdc[27518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 14:57:10.753492 27599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a577685fbf94d6ebf7e2f3d70a85a9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39907 } }
I20260620 14:57:10.753872 27599 leader_election.cc:302] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a577685fbf94d6ebf7e2f3d70a85a9c; no voters:
I20260620 14:57:10.754240 27599 leader_election.cc:288] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:10.754307 27604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:10.754545 27604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [term 1 LEADER]: Becoming Leader. State: Replica: 5a577685fbf94d6ebf7e2f3d70a85a9c, State: Running, Role: LEADER
I20260620 14:57:10.754887 27604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a577685fbf94d6ebf7e2f3d70a85a9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39907 } }
I20260620 14:57:10.755633 27599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:10.756569 27606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a577685fbf94d6ebf7e2f3d70a85a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a577685fbf94d6ebf7e2f3d70a85a9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39907 } } }
I20260620 14:57:10.756714 27606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:10.757016 27605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a577685fbf94d6ebf7e2f3d70a85a9c. Latest consensus state: current_term: 1 leader_uuid: "5a577685fbf94d6ebf7e2f3d70a85a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a577685fbf94d6ebf7e2f3d70a85a9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39907 } } }
I20260620 14:57:10.757190 27605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:10.758389 27612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:10.759320 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:10.743805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35311 (local address 127.0.0.1:39907)
0620 14:57:10.744277 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:10.744292 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:10.744334 (+ 42us) server_negotiation.cc:407] Connection header received
0620 14:57:10.745261 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:10.745292 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:10.745727 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:10.746087 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:10.746962 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:10.748423 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:10.749304 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:10.750119 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:10.750288 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:10.753277 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:10.753283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:10.753296 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:10.753330 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:10.755532 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:10.756155 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:10.756161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:10.756167 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:10.756308 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:10.756720 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:10.756723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:10.756726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:10.757218 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:10.757440 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:10.758268 (+ 828us) server_negotiation.cc:299] Negotiation successful
0620 14:57:10.758534 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":163,"threads_started":1}
I20260620 14:57:10.761675 27612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:10.762192 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:10.767792 27612 catalog_manager.cc:1347] Generated new cluster ID: 5c23ed3fa1ad41df895f17e5fae60488
I20260620 14:57:10.767869 27612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:10.774379 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:10.764495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59443 (local address 127.0.0.1:39907)
0620 14:57:10.764599 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:10.764605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:10.766482 (+ 1877us) server_negotiation.cc:407] Connection header received
0620 14:57:10.766623 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:10.766626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:10.766675 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:10.766762 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:10.767657 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:10.768272 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:10.769090 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:10.769352 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:10.769459 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:10.771027 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:10.771031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:10.771034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:10.771059 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:10.772552 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:10.773039 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:10.773042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:10.773044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:10.773125 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:10.773464 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:10.773468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:10.773469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:10.773644 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:10.773803 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:10.774099 (+ 296us) server_negotiation.cc:299] Negotiation successful
0620 14:57:10.774191 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 14:57:10.779044 27612 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:10.779984 27612 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:10.784590 27612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a577685fbf94d6ebf7e2f3d70a85a9c: Generated new TSK 0
I20260620 14:57:10.785246 27612 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:10.793880 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 27531
W20260620 14:57:10.799621 27516 connection.cc:535] client connection to 127.0.0.1:39907 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39907): Connection reset by peer (error 104)
2026-06-20T14:57:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:10 dist-test-slave-sms9 krb5kdc[27630](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:10 dist-test-slave-sms9 krb5kdc[27630](info): set up 2 sockets
Jun 20 14:57:10 dist-test-slave-sms9 krb5kdc[27630](info): commencing operation
krb5kdc: starting...
W20260620 14:57:11.836643 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:11 dist-test-slave-sms9 krb5kdc[27630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:11.969627 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:12.082937 27643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:12.083184 27643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:12.083225 27643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:12.087147 27643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:12.087203 27643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:12.087221 27643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:12.087237 27643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:12.087252 27643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:12.092311 27643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39269
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:12.093271 27643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:12.094300 27643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:12.103017 27648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:12.103027 27649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:12.103073 27643 server_base.cc:1034] running on GCE node
W20260620 14:57:12.103035 27651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:12.103868 27643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:12.105145 27643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:12.106331 27643 hybrid_clock.cc:648] HybridClock initialized: now 1781967432106311 us; error 38 us; skew 500 ppm
Jun 20 14:57:12 dist-test-slave-sms9 krb5kdc[27630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967432, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:12.109318 27643 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:12.110311 27643 webserver.cc:460] Webserver started at http://127.0.0.1:40347/ using document root <none> and password file <none>
I20260620 14:57:12.110960 27643 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:12.111056 27643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:12.111313 27643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:12.113479 27643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "3359fc9afd4b4c1c880d5ca1638d3e56"
format_stamp: "Formatted at 2026-06-20 14:57:12 on dist-test-slave-sms9"
server_key: "1ad5f9b73ca6a791385937291ca5a852"
server_key_iv: "cfd11995df33fb9a221828bed6ec0bb3"
server_key_version: "encryptionkey@0"
I20260620 14:57:12.114078 27643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "3359fc9afd4b4c1c880d5ca1638d3e56"
format_stamp: "Formatted at 2026-06-20 14:57:12 on dist-test-slave-sms9"
server_key: "1ad5f9b73ca6a791385937291ca5a852"
server_key_iv: "cfd11995df33fb9a221828bed6ec0bb3"
server_key_version: "encryptionkey@0"
I20260620 14:57:12.117969 27643 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:57:12.120637 27658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:12.121968 27643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:57:12.122148 27643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "3359fc9afd4b4c1c880d5ca1638d3e56"
format_stamp: "Formatted at 2026-06-20 14:57:12 on dist-test-slave-sms9"
server_key: "1ad5f9b73ca6a791385937291ca5a852"
server_key_iv: "cfd11995df33fb9a221828bed6ec0bb3"
server_key_version: "encryptionkey@0"
I20260620 14:57:12.122282 27643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:12.145440 27643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:12.148921 27643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:12.149132 27643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:12.156327 27643 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39269
I20260620 14:57:12.156354 27710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39269 every 8 connection(s)
I20260620 14:57:12.157265 27643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:12.159996 27711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:12.165401 27711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56: Bootstrap starting.
I20260620 14:57:12.166234 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 27643
I20260620 14:57:12.166383 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:12.166790 7267 external_mini_cluster.cc:1428] Setting key 30ffd39d168c8dbb12731d03368f8278
I20260620 14:57:12.168457 27711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:12.169354 27711 log.cc:824] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:12.171401 27711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56: No bootstrap required, opened a new log
I20260620 14:57:12.175709 27711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3359fc9afd4b4c1c880d5ca1638d3e56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39269 } }
I20260620 14:57:12.175956 27711 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:12.176046 27711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3359fc9afd4b4c1c880d5ca1638d3e56, State: Initialized, Role: FOLLOWER
I20260620 14:57:12.176440 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:12.168124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51687 (local address 127.0.0.1:39269)
0620 14:57:12.168622 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:12.168645 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:12.168686 (+ 41us) server_negotiation.cc:407] Connection header received
0620 14:57:12.169645 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:12.169687 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:12.170095 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:12.170365 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:12.170757 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:12.171751 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:12.173002 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:12.173525 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:12.173648 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:12.174654 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:12.174659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:12.174671 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:12.175060 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:12.175114 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:12.175137 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:12.175261 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:12.175443 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 14:57:12.175603 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":113,"threads_started":1}
I20260620 14:57:12.176565 27711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3359fc9afd4b4c1c880d5ca1638d3e56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39269 } }
I20260620 14:57:12.176730 27711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:12.176797 27711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:12.176892 27711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:12.177883 27711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3359fc9afd4b4c1c880d5ca1638d3e56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39269 } }
I20260620 14:57:12.178236 27711 leader_election.cc:302] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3359fc9afd4b4c1c880d5ca1638d3e56; no voters:
I20260620 14:57:12.178602 27711 leader_election.cc:288] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:12.178864 27716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:12.179064 27716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [term 1 LEADER]: Becoming Leader. State: Replica: 3359fc9afd4b4c1c880d5ca1638d3e56, State: Running, Role: LEADER
I20260620 14:57:12.179353 27716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3359fc9afd4b4c1c880d5ca1638d3e56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39269 } }
I20260620 14:57:12.179425 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:12.179924 27711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:12.181932 27718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3359fc9afd4b4c1c880d5ca1638d3e56. Latest consensus state: current_term: 1 leader_uuid: "3359fc9afd4b4c1c880d5ca1638d3e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3359fc9afd4b4c1c880d5ca1638d3e56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39269 } } }
I20260620 14:57:12.182061 27718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:12.182255 27717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3359fc9afd4b4c1c880d5ca1638d3e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3359fc9afd4b4c1c880d5ca1638d3e56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39269 } } }
I20260620 14:57:12.182637 27717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56 [sys.catalog]: This master's current role is: LEADER
W20260620 14:57:12.185235 27736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:57:12.185325 27736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:57:12.185391 27737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:12.186091 27737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:12.186321 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:12.181994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39447 (local address 127.0.0.1:39269)
0620 14:57:12.182097 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:12.182102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:12.182398 (+ 296us) server_negotiation.cc:407] Connection header received
0620 14:57:12.182626 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:12.182630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:12.182684 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:12.182738 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:12.183085 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:12.183748 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:12.184595 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:12.184830 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:12.184949 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:12.185843 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:12.185847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:12.185850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:12.185862 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:12.185899 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:12.185910 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:12.186009 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:12.186130 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 14:57:12.186163 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 14:57:12.191862 27737 catalog_manager.cc:1347] Generated new cluster ID: 59a1b387e83f4a42ba33613bc2cd473c
I20260620 14:57:12.191942 27737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:12.209131 27737 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:12.210170 27737 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:12.221079 27737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3359fc9afd4b4c1c880d5ca1638d3e56: Generated new TSK 0
I20260620 14:57:12.221863 27737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:12.232542 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 27643
W20260620 14:57:12.238054 27628 connection.cc:535] client connection to 127.0.0.1:39269 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39269): Connection reset by peer (error 104)
2026-06-20T14:57:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:12 dist-test-slave-sms9 krb5kdc[27742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:12 dist-test-slave-sms9 krb5kdc[27742](info): set up 2 sockets
Jun 20 14:57:12 dist-test-slave-sms9 krb5kdc[27742](info): commencing operation
krb5kdc: starting...
W20260620 14:57:13.282775 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:13 dist-test-slave-sms9 krb5kdc[27742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:13.407855 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:13.520263 27755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:13.520578 27755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:13.520669 27755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:13.525274 27755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:13.525353 27755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:13.525399 27755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:13.525435 27755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:13.525471 27755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:13.530840 27755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37001
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:13.531904 27755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:13.532914 27755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:13.541781 27761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:13.541822 27763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:13.541824 27755 server_base.cc:1034] running on GCE node
W20260620 14:57:13.541855 27760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:13.542758 27755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:13.543848 27755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:13.545019 27755 hybrid_clock.cc:648] HybridClock initialized: now 1781967433544989 us; error 42 us; skew 500 ppm
Jun 20 14:57:13 dist-test-slave-sms9 krb5kdc[27742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967433, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:13.547901 27755 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:13.548785 27755 webserver.cc:460] Webserver started at http://127.0.0.1:34791/ using document root <none> and password file <none>
I20260620 14:57:13.549300 27755 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:13.549360 27755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:13.549523 27755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:13.551571 27755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "ac6e6d4b1cde4511a97f9547603cabd1"
format_stamp: "Formatted at 2026-06-20 14:57:13 on dist-test-slave-sms9"
server_key: "a5dfe2fb23eb0bb9e5dae68b27139968"
server_key_iv: "cd5bf053a5d52accc93d740ac0feff03"
server_key_version: "encryptionkey@0"
I20260620 14:57:13.552075 27755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "ac6e6d4b1cde4511a97f9547603cabd1"
format_stamp: "Formatted at 2026-06-20 14:57:13 on dist-test-slave-sms9"
server_key: "a5dfe2fb23eb0bb9e5dae68b27139968"
server_key_iv: "cd5bf053a5d52accc93d740ac0feff03"
server_key_version: "encryptionkey@0"
I20260620 14:57:13.555753 27755 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:57:13.558106 27770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:13.559351 27755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:57:13.559526 27755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "ac6e6d4b1cde4511a97f9547603cabd1"
format_stamp: "Formatted at 2026-06-20 14:57:13 on dist-test-slave-sms9"
server_key: "a5dfe2fb23eb0bb9e5dae68b27139968"
server_key_iv: "cd5bf053a5d52accc93d740ac0feff03"
server_key_version: "encryptionkey@0"
I20260620 14:57:13.559641 27755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:13.564163 27755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:13.567875 27755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:13.568094 27755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:13.576103 27755 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37001
I20260620 14:57:13.576130 27822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37001 every 8 connection(s)
I20260620 14:57:13.577096 27755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:13.579865 27823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:13.583570 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 27755
I20260620 14:57:13.583714 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:13.584036 7267 external_mini_cluster.cc:1428] Setting key 8ff5c8d109c12193cff0cca10d39b342
I20260620 14:57:13.585634 27823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1: Bootstrap starting.
I20260620 14:57:13.588337 27823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:13.589077 27823 log.cc:824] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:13.590970 27823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1: No bootstrap required, opened a new log
I20260620 14:57:13.593711 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:13.585537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45387 (local address 127.0.0.1:37001)
0620 14:57:13.586012 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:13.586029 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:13.586070 (+ 41us) server_negotiation.cc:407] Connection header received
0620 14:57:13.586940 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:13.586965 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:13.587306 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:13.587667 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:13.592223 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:13.592229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:13.592248 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:13.592262 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:13.592352 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:13.592381 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:13.592507 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:13.592635 (+ 128us) server_negotiation.cc:299] Negotiation successful
0620 14:57:13.592795 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":150,"threads_started":1}
I20260620 14:57:13.595319 27823 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6e6d4b1cde4511a97f9547603cabd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } }
I20260620 14:57:13.595563 27823 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:13.595638 27823 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6e6d4b1cde4511a97f9547603cabd1, State: Initialized, Role: FOLLOWER
I20260620 14:57:13.596041 27823 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6e6d4b1cde4511a97f9547603cabd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } }
I20260620 14:57:13.596215 27823 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:13.596287 27823 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:13.596405 27823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:13.596853 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:13.597434 27823 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6e6d4b1cde4511a97f9547603cabd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } }
I20260620 14:57:13.597767 27823 leader_election.cc:302] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac6e6d4b1cde4511a97f9547603cabd1; no voters:
I20260620 14:57:13.598170 27823 leader_election.cc:288] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:13.598348 27829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:13.598690 27829 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [term 1 LEADER]: Becoming Leader. State: Replica: ac6e6d4b1cde4511a97f9547603cabd1, State: Running, Role: LEADER
I20260620 14:57:13.599121 27829 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6e6d4b1cde4511a97f9547603cabd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } }
I20260620 14:57:13.599586 27823 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:13.600975 27832 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac6e6d4b1cde4511a97f9547603cabd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6e6d4b1cde4511a97f9547603cabd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } } }
I20260620 14:57:13.601112 27832 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:13.601533 27834 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6e6d4b1cde4511a97f9547603cabd1. Latest consensus state: current_term: 1 leader_uuid: "ac6e6d4b1cde4511a97f9547603cabd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6e6d4b1cde4511a97f9547603cabd1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } } }
I20260620 14:57:13.601650 27834 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:13.601684 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:13.599230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46599 (local address 127.0.0.1:37001)
0620 14:57:13.599349 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:13.599353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:13.599756 (+ 403us) server_negotiation.cc:407] Connection header received
0620 14:57:13.599902 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:13.599905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:13.599945 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:13.600039 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:13.601064 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:13.601068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:13.601071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:13.601086 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:13.601119 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:13.601138 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:13.601207 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:13.601438 (+ 231us) server_negotiation.cc:299] Negotiation successful
0620 14:57:13.601495 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 14:57:13.602331 27844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:13.604835 27844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:13.610062 27844 catalog_manager.cc:1347] Generated new cluster ID: 0326bc5a10ad45d8ae4ef9cef69414be
I20260620 14:57:13.610150 27844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:13.629994 27844 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:13.630342 27844 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:13.636960 27844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac6e6d4b1cde4511a97f9547603cabd1: Generated new TSK 0
I20260620 14:57:13.637611 27844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:13.644685 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 27755
2026-06-20T14:57:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:13 dist-test-slave-sms9 krb5kdc[27854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:13 dist-test-slave-sms9 krb5kdc[27854](info): set up 2 sockets
Jun 20 14:57:13 dist-test-slave-sms9 krb5kdc[27854](info): commencing operation
krb5kdc: starting...
W20260620 14:57:14.690747 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:14 dist-test-slave-sms9 krb5kdc[27854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:14.817839 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:14.930197 27867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:14.930603 27867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:14.930655 27867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:14.934722 27867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:14.934794 27867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:14.934815 27867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:14.934832 27867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:14.934849 27867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:14.940364 27867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:14.941440 27867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:14.942448 27867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:57:14.951189 27867 server_base.cc:1034] running on GCE node
W20260620 14:57:14.951143 27873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:14.951109 27872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:14.951109 27875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:14.952031 27867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:14.953150 27867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:14.954317 27867 hybrid_clock.cc:648] HybridClock initialized: now 1781967434954304 us; error 28 us; skew 500 ppm
Jun 20 14:57:14 dist-test-slave-sms9 krb5kdc[27854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:14.957167 27867 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:14.958107 27867 webserver.cc:460] Webserver started at http://127.0.0.1:41567/ using document root <none> and password file <none>
I20260620 14:57:14.958698 27867 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:14.958761 27867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:14.958930 27867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:14.960932 27867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "17a35cdafa7e4b9ca8cfa0837d6d1a22"
format_stamp: "Formatted at 2026-06-20 14:57:14 on dist-test-slave-sms9"
server_key: "4a3a9c12218ea5b69691060f4daf0ebe"
server_key_iv: "d848a287e3004183b21ab989e8e8ca36"
server_key_version: "encryptionkey@0"
I20260620 14:57:14.961448 27867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "17a35cdafa7e4b9ca8cfa0837d6d1a22"
format_stamp: "Formatted at 2026-06-20 14:57:14 on dist-test-slave-sms9"
server_key: "4a3a9c12218ea5b69691060f4daf0ebe"
server_key_iv: "d848a287e3004183b21ab989e8e8ca36"
server_key_version: "encryptionkey@0"
I20260620 14:57:14.965298 27867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:57:14.967818 27882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:14.969151 27867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:57:14.969339 27867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "17a35cdafa7e4b9ca8cfa0837d6d1a22"
format_stamp: "Formatted at 2026-06-20 14:57:14 on dist-test-slave-sms9"
server_key: "4a3a9c12218ea5b69691060f4daf0ebe"
server_key_iv: "d848a287e3004183b21ab989e8e8ca36"
server_key_version: "encryptionkey@0"
I20260620 14:57:14.969456 27867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:14.980338 27867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:14.983839 27867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:14.984061 27867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:14.991525 27867 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40103
I20260620 14:57:14.991562 27934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40103 every 8 connection(s)
I20260620 14:57:14.992478 27867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:14.994765 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 27867
I20260620 14:57:14.994915 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:14.995302 7267 external_mini_cluster.cc:1428] Setting key 6010b6380ba48f9cbcbb2c2567852494
I20260620 14:57:14.995765 27935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:15.002574 27935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22: Bootstrap starting.
Jun 20 14:57:15 dist-test-slave-sms9 krb5kdc[27854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 14:57:15.005090 27935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:15.006095 27935 log.cc:824] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:15.008172 27935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22: No bootstrap required, opened a new log
I20260620 14:57:15.009658 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:14.996845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51379 (local address 127.0.0.1:40103)
0620 14:57:14.997393 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:14.997409 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:14.997446 (+ 37us) server_negotiation.cc:407] Connection header received
0620 14:57:14.998086 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:14.998102 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:14.998442 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:14.998896 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:14.999692 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:15.000663 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:15.001500 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:15.002059 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:15.002239 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:15.004857 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:15.004862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:15.004875 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:15.004903 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:15.006950 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:15.007415 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:15.007420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:15.007424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:15.007530 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:15.007810 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:15.007816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:15.007821 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:15.008205 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:15.008414 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:15.008675 (+ 261us) server_negotiation.cc:299] Negotiation successful
0620 14:57:15.008969 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":173,"threads_started":1}
I20260620 14:57:15.012665 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:15.012480 27935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a35cdafa7e4b9ca8cfa0837d6d1a22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40103 } }
I20260620 14:57:15.012809 27935 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:15.012899 27935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a35cdafa7e4b9ca8cfa0837d6d1a22, State: Initialized, Role: FOLLOWER
I20260620 14:57:15.013324 27935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a35cdafa7e4b9ca8cfa0837d6d1a22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40103 } }
I20260620 14:57:15.013484 27935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:15.013559 27935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:15.013664 27935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:15.014978 27935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a35cdafa7e4b9ca8cfa0837d6d1a22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40103 } }
I20260620 14:57:15.015406 27935 leader_election.cc:302] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17a35cdafa7e4b9ca8cfa0837d6d1a22; no voters:
I20260620 14:57:15.015767 27935 leader_election.cc:288] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:15.015897 27945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:15.016139 27945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [term 1 LEADER]: Becoming Leader. State: Replica: 17a35cdafa7e4b9ca8cfa0837d6d1a22, State: Running, Role: LEADER
I20260620 14:57:15.016556 27945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a35cdafa7e4b9ca8cfa0837d6d1a22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40103 } }
I20260620 14:57:15.017115 27935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:15.017747 27945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17a35cdafa7e4b9ca8cfa0837d6d1a22. Latest consensus state: current_term: 1 leader_uuid: "17a35cdafa7e4b9ca8cfa0837d6d1a22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a35cdafa7e4b9ca8cfa0837d6d1a22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40103 } } }
I20260620 14:57:15.018452 27945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:15.018344 27947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17a35cdafa7e4b9ca8cfa0837d6d1a22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17a35cdafa7e4b9ca8cfa0837d6d1a22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40103 } } }
I20260620 14:57:15.018778 27947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:15.019935 27954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:15.023859 27954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:15.024463 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:15.014826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40703 (local address 127.0.0.1:40103)
0620 14:57:15.014961 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:15.014965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:15.015148 (+ 183us) server_negotiation.cc:407] Connection header received
0620 14:57:15.015309 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:15.015313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:15.015362 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:15.015464 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:15.016375 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:15.017161 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:15.018375 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:15.018762 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:15.018865 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:15.020731 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:15.020735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:15.020738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:15.020765 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:15.022535 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:15.023043 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:15.023047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:15.023049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:15.023148 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:15.023484 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:15.023487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:15.023489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:15.023736 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:15.023880 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:15.024158 (+ 278us) server_negotiation.cc:299] Negotiation successful
0620 14:57:15.024286 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 14:57:15.030435 27954 catalog_manager.cc:1347] Generated new cluster ID: 3c7e6398d9d44a0c873e93b899f199d1
I20260620 14:57:15.030534 27954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:15.051277 27954 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:15.052597 27954 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:15.068388 27954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17a35cdafa7e4b9ca8cfa0837d6d1a22: Generated new TSK 0
I20260620 14:57:15.069176 27954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:15.079078 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 27867
2026-06-20T14:57:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:15 dist-test-slave-sms9 krb5kdc[27966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:15 dist-test-slave-sms9 krb5kdc[27966](info): set up 2 sockets
Jun 20 14:57:15 dist-test-slave-sms9 krb5kdc[27966](info): commencing operation
krb5kdc: starting...
W20260620 14:57:16.126778 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:16 dist-test-slave-sms9 krb5kdc[27966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:16.254256 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40865
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:16.365621 27979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:16.365967 27979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:16.366055 27979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:16.370090 27979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:16.370196 27979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:16.370235 27979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:16.370273 27979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:16.370303 27979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:16.375646 27979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:16.376794 27979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:16.377840 27979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:16.386572 27984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:16.386557 27985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:16.386557 27987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:16.386687 27979 server_base.cc:1034] running on GCE node
I20260620 14:57:16.387434 27979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:16.388540 27979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:16.389736 27979 hybrid_clock.cc:648] HybridClock initialized: now 1781967436389720 us; error 30 us; skew 500 ppm
Jun 20 14:57:16 dist-test-slave-sms9 krb5kdc[27966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967436, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:16.392539 27979 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:16.393455 27979 webserver.cc:460] Webserver started at http://127.0.0.1:35175/ using document root <none> and password file <none>
I20260620 14:57:16.394027 27979 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:16.394122 27979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:16.394364 27979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:16.396486 27979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "a244b9d38593426984ba3c97b35d5adf"
format_stamp: "Formatted at 2026-06-20 14:57:16 on dist-test-slave-sms9"
server_key: "78f3c1dad5ef67f1d8aaaa3627f039df"
server_key_iv: "328a0c16d14f002f9bbd3c7d76ed7679"
server_key_version: "encryptionkey@0"
I20260620 14:57:16.397068 27979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "a244b9d38593426984ba3c97b35d5adf"
format_stamp: "Formatted at 2026-06-20 14:57:16 on dist-test-slave-sms9"
server_key: "78f3c1dad5ef67f1d8aaaa3627f039df"
server_key_iv: "328a0c16d14f002f9bbd3c7d76ed7679"
server_key_version: "encryptionkey@0"
I20260620 14:57:16.401157 27979 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:57:16.403901 27994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:16.405261 27979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:57:16.405416 27979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "a244b9d38593426984ba3c97b35d5adf"
format_stamp: "Formatted at 2026-06-20 14:57:16 on dist-test-slave-sms9"
server_key: "78f3c1dad5ef67f1d8aaaa3627f039df"
server_key_iv: "328a0c16d14f002f9bbd3c7d76ed7679"
server_key_version: "encryptionkey@0"
I20260620 14:57:16.405526 27979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:16.421849 27979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:16.425061 27979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:16.425294 27979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:16.432806 27979 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40865
I20260620 14:57:16.432830 28046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40865 every 8 connection(s)
I20260620 14:57:16.433801 27979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:16.436614 28047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:16.441413 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 27979
I20260620 14:57:16.441620 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:16.441918 7267 external_mini_cluster.cc:1428] Setting key 52d9ebf0ffc54ddbf280801c0dda13f5
I20260620 14:57:16.442196 28047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf: Bootstrap starting.
I20260620 14:57:16.444770 28047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:16.445685 28047 log.cc:824] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:16.447935 28047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf: No bootstrap required, opened a new log
I20260620 14:57:16.451465 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:16.443325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60063 (local address 127.0.0.1:40865)
0620 14:57:16.443784 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:16.443803 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:16.443838 (+ 35us) server_negotiation.cc:407] Connection header received
0620 14:57:16.444490 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:16.444507 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:16.444851 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:16.445179 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:16.445521 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:16.446783 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:16.447956 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:16.448483 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:16.448605 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:16.449500 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:16.449505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:16.449520 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:16.449960 (+ 440us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:16.450024 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:16.450062 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:16.450166 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:16.450442 (+ 276us) server_negotiation.cc:299] Negotiation successful
0620 14:57:16.450707 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":159,"threads_started":1}
I20260620 14:57:16.452092 28047 raft_consensus.cc:348] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a244b9d38593426984ba3c97b35d5adf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40865 } }
I20260620 14:57:16.452346 28047 raft_consensus.cc:374] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:16.452559 28047 raft_consensus.cc:729] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a244b9d38593426984ba3c97b35d5adf, State: Initialized, Role: FOLLOWER
I20260620 14:57:16.453017 28047 consensus_queue.cc:260] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a244b9d38593426984ba3c97b35d5adf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40865 } }
I20260620 14:57:16.453177 28047 raft_consensus.cc:388] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:16.453263 28047 raft_consensus.cc:482] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:16.453388 28047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:16.454526 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:16.454653 28047 raft_consensus.cc:504] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a244b9d38593426984ba3c97b35d5adf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40865 } }
I20260620 14:57:16.455003 28047 leader_election.cc:302] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a244b9d38593426984ba3c97b35d5adf; no voters:
I20260620 14:57:16.455384 28047 leader_election.cc:288] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:16.455454 28052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:16.455662 28052 raft_consensus.cc:686] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [term 1 LEADER]: Becoming Leader. State: Replica: a244b9d38593426984ba3c97b35d5adf, State: Running, Role: LEADER
I20260620 14:57:16.456022 28052 consensus_queue.cc:237] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a244b9d38593426984ba3c97b35d5adf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40865 } }
I20260620 14:57:16.456718 28047 sys_catalog.cc:564] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:16.457249 28052 sys_catalog.cc:455] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a244b9d38593426984ba3c97b35d5adf. Latest consensus state: current_term: 1 leader_uuid: "a244b9d38593426984ba3c97b35d5adf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a244b9d38593426984ba3c97b35d5adf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40865 } } }
I20260620 14:57:16.457429 28052 sys_catalog.cc:458] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:16.457690 28053 sys_catalog.cc:455] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a244b9d38593426984ba3c97b35d5adf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a244b9d38593426984ba3c97b35d5adf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40865 } } }
I20260620 14:57:16.457808 28053 sys_catalog.cc:458] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:16.457850 28063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:16.461509 28063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:16.466060 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:16.459361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35683 (local address 127.0.0.1:40865)
0620 14:57:16.459509 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:16.459514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:16.459883 (+ 369us) server_negotiation.cc:407] Connection header received
0620 14:57:16.460082 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:16.460086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:16.460152 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:16.460249 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:16.460607 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:16.461374 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:16.462624 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:16.462969 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:16.463088 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:16.465549 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:16.465553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:16.465556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:16.465570 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:16.465607 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:16.465619 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:16.465703 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:16.465831 (+ 128us) server_negotiation.cc:299] Negotiation successful
0620 14:57:16.465889 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 14:57:16.467475 28063 catalog_manager.cc:1347] Generated new cluster ID: 8acef2ce58094c2bad245e1f7ddd013d
I20260620 14:57:16.467540 28063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:16.478533 28063 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:16.479903 28063 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:16.497501 28063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a244b9d38593426984ba3c97b35d5adf: Generated new TSK 0
I20260620 14:57:16.498530 28063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:16.510937 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 27979
2026-06-20T14:57:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:16 dist-test-slave-sms9 krb5kdc[28078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:16 dist-test-slave-sms9 krb5kdc[28078](info): set up 2 sockets
Jun 20 14:57:16 dist-test-slave-sms9 krb5kdc[28078](info): commencing operation
krb5kdc: starting...
W20260620 14:57:17.559340 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:17 dist-test-slave-sms9 krb5kdc[28078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:17.681762 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41739
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:17.799113 28091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:17.799445 28091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:17.799546 28091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:17.804294 28091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:17.804394 28091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:17.804430 28091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:17.804467 28091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:17.804502 28091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:17.810202 28091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:17.811362 28091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:17.812394 28091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:17.821053 28096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:17.821179 28091 server_base.cc:1034] running on GCE node
W20260620 14:57:17.821049 28099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:17.821105 28097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:17.821995 28091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:17.823175 28091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:17.824375 28091 hybrid_clock.cc:648] HybridClock initialized: now 1781967437824361 us; error 25 us; skew 500 ppm
Jun 20 14:57:17 dist-test-slave-sms9 krb5kdc[28078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967437, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:17.827675 28091 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:17.828644 28091 webserver.cc:460] Webserver started at http://127.0.0.1:44179/ using document root <none> and password file <none>
I20260620 14:57:17.829212 28091 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:17.829303 28091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:17.829512 28091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:17.831614 28091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "449b24da0029439091f3ef5e3ff1bc2a"
format_stamp: "Formatted at 2026-06-20 14:57:17 on dist-test-slave-sms9"
server_key: "b867a5492115953b87feeaa50562dfae"
server_key_iv: "5f0f8ef517536782178f7af5000bfd18"
server_key_version: "encryptionkey@0"
I20260620 14:57:17.832204 28091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "449b24da0029439091f3ef5e3ff1bc2a"
format_stamp: "Formatted at 2026-06-20 14:57:17 on dist-test-slave-sms9"
server_key: "b867a5492115953b87feeaa50562dfae"
server_key_iv: "5f0f8ef517536782178f7af5000bfd18"
server_key_version: "encryptionkey@0"
I20260620 14:57:17.836396 28091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260620 14:57:17.839011 28106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:17.840225 28091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:17.840404 28091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "449b24da0029439091f3ef5e3ff1bc2a"
format_stamp: "Formatted at 2026-06-20 14:57:17 on dist-test-slave-sms9"
server_key: "b867a5492115953b87feeaa50562dfae"
server_key_iv: "5f0f8ef517536782178f7af5000bfd18"
server_key_version: "encryptionkey@0"
I20260620 14:57:17.840528 28091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:17.845716 28091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:17.849143 28091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:17.849349 28091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:17.856509 28091 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41739
I20260620 14:57:17.856588 28158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41739 every 8 connection(s)
I20260620 14:57:17.857506 28091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:17.858686 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 28091
I20260620 14:57:17.858830 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:17.859154 7267 external_mini_cluster.cc:1428] Setting key 924d8f630b3fbf11add4c08f2f48f584
I20260620 14:57:17.860803 28159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:17.864765 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:17.860512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55615 (local address 127.0.0.1:41739)
0620 14:57:17.860920 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:17.860939 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:17.860980 (+ 41us) server_negotiation.cc:407] Connection header received
0620 14:57:17.861794 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:17.861816 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:17.862168 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:17.862596 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:17.863339 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:17.863344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:17.863359 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:17.863371 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:17.863444 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:17.863476 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:17.863616 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:17.863686 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 14:57:17.863834 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260620 14:57:17.867491 28159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a: Bootstrap starting.
I20260620 14:57:17.867661 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:17.869892 28159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:17.870862 28159 log.cc:824] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:17.871485 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:17.869730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49303 (local address 127.0.0.1:41739)
0620 14:57:17.869876 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:17.869882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:17.870038 (+ 156us) server_negotiation.cc:407] Connection header received
0620 14:57:17.870216 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:17.870218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:17.870253 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:17.870319 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:17.871050 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:17.871054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:17.871057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:17.871069 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:17.871104 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:17.871115 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:17.871185 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:17.871325 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 14:57:17.871365 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 14:57:17.872918 28159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a: No bootstrap required, opened a new log
I20260620 14:57:17.876441 28159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449b24da0029439091f3ef5e3ff1bc2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } }
I20260620 14:57:17.876670 28159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:17.876760 28159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 449b24da0029439091f3ef5e3ff1bc2a, State: Initialized, Role: FOLLOWER
I20260620 14:57:17.877230 28159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449b24da0029439091f3ef5e3ff1bc2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } }
I20260620 14:57:17.877352 28159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:17.877437 28159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:17.877552 28159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:17.878535 28159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449b24da0029439091f3ef5e3ff1bc2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } }
I20260620 14:57:17.878882 28159 leader_election.cc:302] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 449b24da0029439091f3ef5e3ff1bc2a; no voters:
I20260620 14:57:17.879223 28159 leader_election.cc:288] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:17.879494 28169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:17.879698 28169 raft_consensus.cc:686] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [term 1 LEADER]: Becoming Leader. State: Replica: 449b24da0029439091f3ef5e3ff1bc2a, State: Running, Role: LEADER
I20260620 14:57:17.879974 28169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449b24da0029439091f3ef5e3ff1bc2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } }
I20260620 14:57:17.880527 28159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:17.881547 28170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "449b24da0029439091f3ef5e3ff1bc2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449b24da0029439091f3ef5e3ff1bc2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } } }
I20260620 14:57:17.881673 28170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:17.882004 28171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 449b24da0029439091f3ef5e3ff1bc2a. Latest consensus state: current_term: 1 leader_uuid: "449b24da0029439091f3ef5e3ff1bc2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449b24da0029439091f3ef5e3ff1bc2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41739 } } }
I20260620 14:57:17.882092 28171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:17.882333 28178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:17.885330 28178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:17.890168 28178 catalog_manager.cc:1347] Generated new cluster ID: c09eea087c9741c1bdc9f7ddb6cf6308
I20260620 14:57:17.890228 28178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:17.909570 28178 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:17.909932 28178 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:17.918856 28178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 449b24da0029439091f3ef5e3ff1bc2a: Generated new TSK 0
I20260620 14:57:17.919665 28178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:17.930100 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 28091
2026-06-20T14:57:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:17 dist-test-slave-sms9 krb5kdc[28190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:17 dist-test-slave-sms9 krb5kdc[28190](info): set up 2 sockets
Jun 20 14:57:17 dist-test-slave-sms9 krb5kdc[28190](info): commencing operation
krb5kdc: starting...
W20260620 14:57:18.974726 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:19 dist-test-slave-sms9 krb5kdc[28190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:19.093742 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37943
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:19.205765 28203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:19.206095 28203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:19.206175 28203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:19.210376 28203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:19.210506 28203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:19.210552 28203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:19.210587 28203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:19.210620 28203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:19.216374 28203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:19.217540 28203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:19.218644 28203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:19.227176 28208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:19.227177 28209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:19.227146 28211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:19.227722 28203 server_base.cc:1034] running on GCE node
I20260620 14:57:19.228255 28203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:19.229398 28203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:19.230571 28203 hybrid_clock.cc:648] HybridClock initialized: now 1781967439230542 us; error 39 us; skew 500 ppm
Jun 20 14:57:19 dist-test-slave-sms9 krb5kdc[28190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967439, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:19.237376 28203 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:19.238371 28203 webserver.cc:460] Webserver started at http://127.0.0.1:41099/ using document root <none> and password file <none>
I20260620 14:57:19.238996 28203 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:19.239059 28203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:19.239233 28203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:19.241632 28203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "0c7be2dedaa9429e981e07c7de557f0e"
format_stamp: "Formatted at 2026-06-20 14:57:19 on dist-test-slave-sms9"
server_key: "16dd31b57e562d7e57730f20a525a323"
server_key_iv: "ce574f41894e918beb90d1ad487fd2db"
server_key_version: "encryptionkey@0"
I20260620 14:57:19.242203 28203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "0c7be2dedaa9429e981e07c7de557f0e"
format_stamp: "Formatted at 2026-06-20 14:57:19 on dist-test-slave-sms9"
server_key: "16dd31b57e562d7e57730f20a525a323"
server_key_iv: "ce574f41894e918beb90d1ad487fd2db"
server_key_version: "encryptionkey@0"
I20260620 14:57:19.246121 28203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260620 14:57:19.248903 28218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:19.250319 28203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 14:57:19.250456 28203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "0c7be2dedaa9429e981e07c7de557f0e"
format_stamp: "Formatted at 2026-06-20 14:57:19 on dist-test-slave-sms9"
server_key: "16dd31b57e562d7e57730f20a525a323"
server_key_iv: "ce574f41894e918beb90d1ad487fd2db"
server_key_version: "encryptionkey@0"
I20260620 14:57:19.250603 28203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:19.270661 28203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:19.273761 28203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:19.273983 28203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:19.281754 28203 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37943
I20260620 14:57:19.281783 28270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37943 every 8 connection(s)
I20260620 14:57:19.282828 28203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:19.285562 28271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:19.290566 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 28203
I20260620 14:57:19.290721 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:19.291013 7267 external_mini_cluster.cc:1428] Setting key 3cf71b9f547c07547d59250a8f0f8909
I20260620 14:57:19.291589 28271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e: Bootstrap starting.
I20260620 14:57:19.294077 28271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:19.294879 28271 log.cc:824] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:19.296734 28271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e: No bootstrap required, opened a new log
Jun 20 14:57:19 dist-test-slave-sms9 krb5kdc[28190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 14:57:19.300295 28271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7be2dedaa9429e981e07c7de557f0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37943 } }
I20260620 14:57:19.300545 28271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:19.300628 28271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7be2dedaa9429e981e07c7de557f0e, State: Initialized, Role: FOLLOWER
I20260620 14:57:19.301095 28271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7be2dedaa9429e981e07c7de557f0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37943 } }
I20260620 14:57:19.301241 28271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:19.301327 28271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:19.301443 28271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:19.302337 28271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7be2dedaa9429e981e07c7de557f0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37943 } }
I20260620 14:57:19.302706 28271 leader_election.cc:302] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c7be2dedaa9429e981e07c7de557f0e; no voters:
I20260620 14:57:19.303032 28271 leader_election.cc:288] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:19.303145 28276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:19.303429 28276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [term 1 LEADER]: Becoming Leader. State: Replica: 0c7be2dedaa9429e981e07c7de557f0e, State: Running, Role: LEADER
I20260620 14:57:19.303803 28276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7be2dedaa9429e981e07c7de557f0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37943 } }
I20260620 14:57:19.304332 28271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:19.305022 28276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c7be2dedaa9429e981e07c7de557f0e. Latest consensus state: current_term: 1 leader_uuid: "0c7be2dedaa9429e981e07c7de557f0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7be2dedaa9429e981e07c7de557f0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37943 } } }
I20260620 14:57:19.305156 28276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:19.305573 28277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c7be2dedaa9429e981e07c7de557f0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7be2dedaa9429e981e07c7de557f0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37943 } } }
I20260620 14:57:19.305688 28277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:19.306335 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:19.292445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:56127 (local address 127.0.0.1:37943)
0620 14:57:19.292913 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:19.292932 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:19.292968 (+ 36us) server_negotiation.cc:407] Connection header received
0620 14:57:19.293611 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:19.293628 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:19.293980 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:19.294320 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:19.295209 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:19.296203 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:19.297131 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:19.297601 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:19.297779 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:19.300344 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:19.300349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:19.300363 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:19.300393 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:19.302214 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:19.302843 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:19.302850 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:19.302856 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:19.303029 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:19.303477 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:19.303481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:19.303485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:19.304014 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:19.304229 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:19.304949 (+ 720us) server_negotiation.cc:299] Negotiation successful
0620 14:57:19.305211 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":137,"threads_started":1}
I20260620 14:57:19.306730 28283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:19.309585 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:19.310232 28283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:19.316416 28283 catalog_manager.cc:1347] Generated new cluster ID: ce039e49a57347dc8bddaa8bff6bfe53
I20260620 14:57:19.316500 28283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:19.326067 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:19.312795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:36169 (local address 127.0.0.1:37943)
0620 14:57:19.312898 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:19.312902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:19.313170 (+ 268us) server_negotiation.cc:407] Connection header received
0620 14:57:19.313293 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:19.313296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:19.313347 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:19.313488 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:19.314338 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:19.315233 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:19.319174 (+ 3941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:19.319782 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:19.319905 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:19.321834 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:19.321838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:19.321841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:19.321877 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:19.323855 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:19.324491 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:19.324495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:19.324496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:19.324610 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:19.325051 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:19.325057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:19.325058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:19.325354 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:19.325514 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:19.325793 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 14:57:19.325894 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 14:57:19.337432 28283 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:19.338445 28283 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:19.345798 28283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c7be2dedaa9429e981e07c7de557f0e: Generated new TSK 0
I20260620 14:57:19.346699 28283 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:19.358052 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 28203
W20260620 14:57:19.363374 28188 connection.cc:535] client connection to 127.0.0.1:37943 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37943): Connection reset by peer (error 104)
2026-06-20T14:57:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:19 dist-test-slave-sms9 krb5kdc[28302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:19 dist-test-slave-sms9 krb5kdc[28302](info): set up 2 sockets
Jun 20 14:57:19 dist-test-slave-sms9 krb5kdc[28302](info): commencing operation
krb5kdc: starting...
W20260620 14:57:20.402742 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:20 dist-test-slave-sms9 krb5kdc[28302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:20.525178 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:20.637143 28315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:20.637451 28315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:20.637496 28315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:20.641402 28315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:20.641463 28315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:20.641482 28315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:20.641499 28315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:20.641515 28315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:20.646808 28315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:20.647791 28315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:20.648742 28315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:20.657132 28323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:20.657142 28321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:20.657184 28315 server_base.cc:1034] running on GCE node
W20260620 14:57:20.657142 28320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:20.657958 28315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:20.659389 28315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:20.660589 28315 hybrid_clock.cc:648] HybridClock initialized: now 1781967440660561 us; error 43 us; skew 500 ppm
Jun 20 14:57:20 dist-test-slave-sms9 krb5kdc[28302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967440, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:20.663305 28315 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:20.664258 28315 webserver.cc:460] Webserver started at http://127.0.0.1:42403/ using document root <none> and password file <none>
I20260620 14:57:20.664778 28315 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:20.664836 28315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:20.665035 28315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:20.667164 28315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "50a6a72de432421482a079b5f2daa046"
format_stamp: "Formatted at 2026-06-20 14:57:20 on dist-test-slave-sms9"
server_key: "4c317e7c8403c21434bf24e7c11c041c"
server_key_iv: "b14a601a848f3e2e380969563a570510"
server_key_version: "encryptionkey@0"
I20260620 14:57:20.667665 28315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "50a6a72de432421482a079b5f2daa046"
format_stamp: "Formatted at 2026-06-20 14:57:20 on dist-test-slave-sms9"
server_key: "4c317e7c8403c21434bf24e7c11c041c"
server_key_iv: "b14a601a848f3e2e380969563a570510"
server_key_version: "encryptionkey@0"
I20260620 14:57:20.671288 28315 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:57:20.673625 28330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:20.674844 28315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:20.674963 28315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "50a6a72de432421482a079b5f2daa046"
format_stamp: "Formatted at 2026-06-20 14:57:20 on dist-test-slave-sms9"
server_key: "4c317e7c8403c21434bf24e7c11c041c"
server_key_iv: "b14a601a848f3e2e380969563a570510"
server_key_version: "encryptionkey@0"
I20260620 14:57:20.675050 28315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:20.690886 28315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:20.693653 28315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:20.693809 28315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:20.701429 28315 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37329
I20260620 14:57:20.701455 28382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37329 every 8 connection(s)
I20260620 14:57:20.702389 28315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:20.705161 28383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:20.710923 28383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046: Bootstrap starting.
I20260620 14:57:20.711704 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 28315
I20260620 14:57:20.711862 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:20.712169 7267 external_mini_cluster.cc:1428] Setting key 661b5456ae29e83e1e950ecdeb362e36
I20260620 14:57:20.713793 28383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:20.714605 28383 log.cc:824] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:20.716634 28383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046: No bootstrap required, opened a new log
I20260620 14:57:20.720420 28383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a6a72de432421482a079b5f2daa046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } }
I20260620 14:57:20.720775 28383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:20.720863 28383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50a6a72de432421482a079b5f2daa046, State: Initialized, Role: FOLLOWER
I20260620 14:57:20.721305 28383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a6a72de432421482a079b5f2daa046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } }
I20260620 14:57:20.721462 28383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:20.721516 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:20.713299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:53511 (local address 127.0.0.1:37329)
0620 14:57:20.713749 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:20.713763 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:20.713797 (+ 34us) server_negotiation.cc:407] Connection header received
0620 14:57:20.714623 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:20.714642 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:20.715034 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:20.715382 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:20.715873 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:20.716895 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:20.717801 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:20.718292 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:20.718541 (+ 249us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:20.719593 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:20.719599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:20.719613 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:20.720028 (+ 415us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:20.720085 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:20.720107 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:20.720250 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:20.720431 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 14:57:20.720573 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":128,"threads_started":1}
I20260620 14:57:20.721550 28383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:20.721736 28383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:20.722906 28383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a6a72de432421482a079b5f2daa046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } }
I20260620 14:57:20.723263 28383 leader_election.cc:302] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50a6a72de432421482a079b5f2daa046; no voters:
I20260620 14:57:20.723630 28383 leader_election.cc:288] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:20.723723 28388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:20.723974 28388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [term 1 LEADER]: Becoming Leader. State: Replica: 50a6a72de432421482a079b5f2daa046, State: Running, Role: LEADER
I20260620 14:57:20.724345 28388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a6a72de432421482a079b5f2daa046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } }
I20260620 14:57:20.724675 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:20.724916 28383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:20.726086 28389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50a6a72de432421482a079b5f2daa046" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a6a72de432421482a079b5f2daa046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } } }
I20260620 14:57:20.726238 28389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:20.726585 28390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50a6a72de432421482a079b5f2daa046. Latest consensus state: current_term: 1 leader_uuid: "50a6a72de432421482a079b5f2daa046" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a6a72de432421482a079b5f2daa046" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } } }
I20260620 14:57:20.726665 28390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:20.727711 28401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:20.730199 28401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:20.735224 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:20.727076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:57105 (local address 127.0.0.1:37329)
0620 14:57:20.727171 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:20.727175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:20.727669 (+ 494us) server_negotiation.cc:407] Connection header received
0620 14:57:20.730668 (+ 2999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:20.730672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:20.730724 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:20.730778 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:20.731237 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:20.731997 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:20.733227 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:20.733559 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:20.733668 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:20.734679 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:20.734683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:20.734686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:20.734699 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:20.734729 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:20.734742 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:20.734842 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:20.735018 (+ 176us) server_negotiation.cc:299] Negotiation successful
0620 14:57:20.735056 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 14:57:20.736708 28401 catalog_manager.cc:1347] Generated new cluster ID: 69cc97e8435740e9b81e69c2aaa486ec
I20260620 14:57:20.736776 28401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:20.748401 28401 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:20.749536 28401 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:20.757043 28401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50a6a72de432421482a079b5f2daa046: Generated new TSK 0
I20260620 14:57:20.757714 28401 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:20.761214 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 28315
W20260620 14:57:20.766434 28300 connection.cc:535] client connection to 127.0.0.1:37329 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37329): Connection reset by peer (error 104)
2026-06-20T14:57:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:20 dist-test-slave-sms9 krb5kdc[28414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:20 dist-test-slave-sms9 krb5kdc[28414](info): set up 2 sockets
Jun 20 14:57:20 dist-test-slave-sms9 krb5kdc[28414](info): commencing operation
krb5kdc: starting...
W20260620 14:57:21.806748 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:21 dist-test-slave-sms9 krb5kdc[28414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:21.937158 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41847
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:22.048837 28427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:22.049113 28427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:22.049153 28427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:22.053198 28427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:22.053263 28427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:22.053282 28427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:22.053299 28427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:22.053356 28427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:22.058645 28427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:22.059731 28427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:22.060745 28427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:22.070034 28435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:22.070099 28427 server_base.cc:1034] running on GCE node
W20260620 14:57:22.070029 28432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:22.070029 28433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:22.070880 28427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:22.072060 28427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:22.073253 28427 hybrid_clock.cc:648] HybridClock initialized: now 1781967442073221 us; error 52 us; skew 500 ppm
Jun 20 14:57:22 dist-test-slave-sms9 krb5kdc[28414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967442, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:22.076143 28427 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:22.077040 28427 webserver.cc:460] Webserver started at http://127.0.0.1:43289/ using document root <none> and password file <none>
I20260620 14:57:22.077608 28427 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:22.077697 28427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:22.077924 28427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:22.079998 28427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "86a87baa6ddc47e2a5b0d4e949452d3e"
format_stamp: "Formatted at 2026-06-20 14:57:22 on dist-test-slave-sms9"
server_key: "d4df8582b168b5503c601ebd25b4a21d"
server_key_iv: "122f2af0af74602369ebf5ac30c9b240"
server_key_version: "encryptionkey@0"
I20260620 14:57:22.080567 28427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "86a87baa6ddc47e2a5b0d4e949452d3e"
format_stamp: "Formatted at 2026-06-20 14:57:22 on dist-test-slave-sms9"
server_key: "d4df8582b168b5503c601ebd25b4a21d"
server_key_iv: "122f2af0af74602369ebf5ac30c9b240"
server_key_version: "encryptionkey@0"
I20260620 14:57:22.084311 28427 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260620 14:57:22.086735 28442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:22.087873 28427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 14:57:22.088017 28427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "86a87baa6ddc47e2a5b0d4e949452d3e"
format_stamp: "Formatted at 2026-06-20 14:57:22 on dist-test-slave-sms9"
server_key: "d4df8582b168b5503c601ebd25b4a21d"
server_key_iv: "122f2af0af74602369ebf5ac30c9b240"
server_key_version: "encryptionkey@0"
I20260620 14:57:22.088132 28427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:22.092746 28427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:22.095713 28427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:22.095908 28427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:22.102572 28427 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41847
I20260620 14:57:22.102596 28494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41847 every 8 connection(s)
I20260620 14:57:22.103515 28427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:22.106272 28495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:22.111806 28495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e: Bootstrap starting.
I20260620 14:57:22.113168 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 28427
I20260620 14:57:22.113346 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:22.113687 7267 external_mini_cluster.cc:1428] Setting key fef5afa89b429f7a164a34970f9e8837
I20260620 14:57:22.114506 28495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:22.115309 28495 log.cc:824] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:22.117358 28495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e: No bootstrap required, opened a new log
I20260620 14:57:22.119411 28498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:22.114995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:44361 (local address 127.0.0.1:41847)
0620 14:57:22.115559 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:22.115574 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:22.115611 (+ 37us) server_negotiation.cc:407] Connection header received
0620 14:57:22.116485 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:22.116505 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:22.116829 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:22.117127 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:22.118002 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:22.118007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:22.118019 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:22.118030 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:22.118087 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:22.118107 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:22.118201 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:22.118349 (+ 148us) server_negotiation.cc:299] Negotiation successful
0620 14:57:22.118585 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":161,"threads_started":1}
I20260620 14:57:22.121573 28495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a87baa6ddc47e2a5b0d4e949452d3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41847 } }
I20260620 14:57:22.121850 28495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:22.121929 28495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a87baa6ddc47e2a5b0d4e949452d3e, State: Initialized, Role: FOLLOWER
I20260620 14:57:22.122390 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:22.122390 28495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a87baa6ddc47e2a5b0d4e949452d3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41847 } }
I20260620 14:57:22.122560 28495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:22.122629 28495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:22.122746 28495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:22.123646 28495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a87baa6ddc47e2a5b0d4e949452d3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41847 } }
I20260620 14:57:22.123973 28495 leader_election.cc:302] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86a87baa6ddc47e2a5b0d4e949452d3e; no voters:
I20260620 14:57:22.124290 28495 leader_election.cc:288] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:22.124392 28504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:22.124599 28504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [term 1 LEADER]: Becoming Leader. State: Replica: 86a87baa6ddc47e2a5b0d4e949452d3e, State: Running, Role: LEADER
I20260620 14:57:22.124876 28504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a87baa6ddc47e2a5b0d4e949452d3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41847 } }
I20260620 14:57:22.125562 28495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:22.126101 28504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86a87baa6ddc47e2a5b0d4e949452d3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a87baa6ddc47e2a5b0d4e949452d3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41847 } } }
I20260620 14:57:22.126283 28504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:22.126531 28498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:22.124481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:54717 (local address 127.0.0.1:41847)
0620 14:57:22.124589 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:22.124593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:22.124837 (+ 244us) server_negotiation.cc:407] Connection header received
0620 14:57:22.124994 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:22.124997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:22.125030 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:22.125120 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:22.125851 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:22.125855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:22.125857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:22.125869 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:22.125895 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:22.125909 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:22.126009 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:22.126341 (+ 332us) server_negotiation.cc:299] Negotiation successful
0620 14:57:22.126381 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 14:57:22.126720 28504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86a87baa6ddc47e2a5b0d4e949452d3e. Latest consensus state: current_term: 1 leader_uuid: "86a87baa6ddc47e2a5b0d4e949452d3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a87baa6ddc47e2a5b0d4e949452d3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41847 } } }
I20260620 14:57:22.126870 28504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:22.127704 28512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:22.130270 28512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:22.135643 28512 catalog_manager.cc:1347] Generated new cluster ID: 478c9175ea4b4dc3869a60505964f6a6
I20260620 14:57:22.135716 28512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:22.154686 28512 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:22.155061 28512 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:22.161407 28512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86a87baa6ddc47e2a5b0d4e949452d3e: Generated new TSK 0
I20260620 14:57:22.162124 28512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:22.168663 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 28427
2026-06-20T14:57:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:22 dist-test-slave-sms9 krb5kdc[28526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:22 dist-test-slave-sms9 krb5kdc[28526](info): set up 2 sockets
Jun 20 14:57:22 dist-test-slave-sms9 krb5kdc[28526](info): commencing operation
krb5kdc: starting...
W20260620 14:57:23.198784 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:23 dist-test-slave-sms9 krb5kdc[28526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:23.322032 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:23.430073 28539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:23.430339 28539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:23.430380 28539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:23.434324 28539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:23.434383 28539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:23.434402 28539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:23.434418 28539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:23.434434 28539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:23.439581 28539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:23.440587 28539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:23.441529 28539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:23.450044 28547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:23.450095 28544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:23.450034 28545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:23.450196 28539 server_base.cc:1034] running on GCE node
I20260620 14:57:23.450887 28539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:23.452031 28539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:23.453267 28539 hybrid_clock.cc:648] HybridClock initialized: now 1781967443453251 us; error 36 us; skew 500 ppm
Jun 20 14:57:23 dist-test-slave-sms9 krb5kdc[28526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967443, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:23.456318 28539 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:23.457244 28539 webserver.cc:460] Webserver started at http://127.0.0.1:34077/ using document root <none> and password file <none>
I20260620 14:57:23.457777 28539 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:23.457839 28539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:23.458009 28539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:23.460155 28539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "3476f589476542dfb3b4f95c1676c2d5"
format_stamp: "Formatted at 2026-06-20 14:57:23 on dist-test-slave-sms9"
server_key: "c2b6e9abcbb8776aabdd4f17432b6b74"
server_key_iv: "d7eb4d25981e9ac0bea2ed83621f495c"
server_key_version: "encryptionkey@0"
I20260620 14:57:23.460700 28539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "3476f589476542dfb3b4f95c1676c2d5"
format_stamp: "Formatted at 2026-06-20 14:57:23 on dist-test-slave-sms9"
server_key: "c2b6e9abcbb8776aabdd4f17432b6b74"
server_key_iv: "d7eb4d25981e9ac0bea2ed83621f495c"
server_key_version: "encryptionkey@0"
I20260620 14:57:23.464613 28539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:57:23.467233 28554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:23.468489 28539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:23.468628 28539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "3476f589476542dfb3b4f95c1676c2d5"
format_stamp: "Formatted at 2026-06-20 14:57:23 on dist-test-slave-sms9"
server_key: "c2b6e9abcbb8776aabdd4f17432b6b74"
server_key_iv: "d7eb4d25981e9ac0bea2ed83621f495c"
server_key_version: "encryptionkey@0"
I20260620 14:57:23.468750 28539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:23.481813 28539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:23.485101 28539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:23.485311 28539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:23.493330 28539 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42477
I20260620 14:57:23.493357 28606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42477 every 8 connection(s)
I20260620 14:57:23.494361 28539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:23.497857 28607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:23.498246 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 28539
I20260620 14:57:23.498404 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:23.498798 7267 external_mini_cluster.cc:1428] Setting key e89cc381e1925d4081f7653d6901415e
I20260620 14:57:23.506563 28607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5: Bootstrap starting.
I20260620 14:57:23.509316 28607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:23.510308 28607 log.cc:824] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5: Log is configured to *not* fsync() on all Append() calls
Jun 20 14:57:23 dist-test-slave-sms9 krb5kdc[28526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 14:57:23.512789 28607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5: No bootstrap required, opened a new log
I20260620 14:57:23.517040 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:23.500616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:54381 (local address 127.0.0.1:42477)
0620 14:57:23.501142 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:23.501157 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:23.501192 (+ 35us) server_negotiation.cc:407] Connection header received
0620 14:57:23.502163 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:23.502189 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:23.502662 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:23.503047 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:23.505486 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:23.506677 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:23.507890 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:23.508379 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:23.508508 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:23.511482 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:23.511488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:23.511505 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:23.511535 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:23.513601 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:23.514160 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:23.514166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:23.514172 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:23.514316 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:23.514723 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:23.514726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:23.514728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:23.515355 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:23.515599 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:23.515815 (+ 216us) server_negotiation.cc:299] Negotiation successful
0620 14:57:23.516060 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":138,"threads_started":1}
I20260620 14:57:23.517529 28607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3476f589476542dfb3b4f95c1676c2d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260620 14:57:23.517781 28607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:23.517840 28607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3476f589476542dfb3b4f95c1676c2d5, State: Initialized, Role: FOLLOWER
I20260620 14:57:23.518309 28607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3476f589476542dfb3b4f95c1676c2d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260620 14:57:23.518529 28607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:23.518599 28607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:23.518698 28607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:23.519812 28607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3476f589476542dfb3b4f95c1676c2d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260620 14:57:23.520159 28607 leader_election.cc:302] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3476f589476542dfb3b4f95c1676c2d5; no voters:
I20260620 14:57:23.520304 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:23.520534 28607 leader_election.cc:288] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:23.520624 28612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:23.520784 28612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [term 1 LEADER]: Becoming Leader. State: Replica: 3476f589476542dfb3b4f95c1676c2d5, State: Running, Role: LEADER
I20260620 14:57:23.521140 28612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3476f589476542dfb3b4f95c1676c2d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260620 14:57:23.521818 28607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:23.522377 28612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3476f589476542dfb3b4f95c1676c2d5. Latest consensus state: current_term: 1 leader_uuid: "3476f589476542dfb3b4f95c1676c2d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3476f589476542dfb3b4f95c1676c2d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } } }
I20260620 14:57:23.522594 28612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:23.522804 28613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3476f589476542dfb3b4f95c1676c2d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3476f589476542dfb3b4f95c1676c2d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } } }
I20260620 14:57:23.522910 28613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:23.523033 28625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:23.526331 28625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:23.531970 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:23.522944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:56251 (local address 127.0.0.1:42477)
0620 14:57:23.523047 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:23.523052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:23.523251 (+ 199us) server_negotiation.cc:407] Connection header received
0620 14:57:23.523453 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:23.523456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:23.523507 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:23.523557 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:23.524483 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:23.525233 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:23.526407 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:23.526771 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:23.526835 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:23.528554 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:23.528558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:23.528561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:23.528586 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:23.530154 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:23.530698 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:23.530704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:23.530706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:23.530820 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:23.531122 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:23.531126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:23.531127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:23.531334 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:23.531482 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:23.531701 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 14:57:23.531806 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 14:57:23.538271 28625 catalog_manager.cc:1347] Generated new cluster ID: 97efe77b30ba456c9029c0737b2b6683
I20260620 14:57:23.538357 28625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:23.545799 28625 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:23.546865 28625 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:23.553399 28625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3476f589476542dfb3b4f95c1676c2d5: Generated new TSK 0
I20260620 14:57:23.554121 28625 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:23.565271 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 28539
W20260620 14:57:23.570755 28524 connection.cc:535] client connection to 127.0.0.1:42477 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42477): Connection reset by peer (error 104)
2026-06-20T14:57:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:23 dist-test-slave-sms9 krb5kdc[28638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:23 dist-test-slave-sms9 krb5kdc[28638](info): set up 2 sockets
Jun 20 14:57:23 dist-test-slave-sms9 krb5kdc[28638](info): commencing operation
krb5kdc: starting...
W20260620 14:57:24.682809 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:24 dist-test-slave-sms9 krb5kdc[28638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:24.809688 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:24.920717 28651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:24.921010 28651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:24.921054 28651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:24.925009 28651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:24.925068 28651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:24.925087 28651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:24.925104 28651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:24.925119 28651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:24.930514 28651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44127
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:24.931571 28651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:24.932595 28651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:24.941335 28657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:24.941335 28659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:24.941380 28656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:24.941833 28651 server_base.cc:1034] running on GCE node
I20260620 14:57:24.942364 28651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:24.943606 28651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:24.944795 28651 hybrid_clock.cc:648] HybridClock initialized: now 1781967444944754 us; error 35 us; skew 500 ppm
Jun 20 14:57:24 dist-test-slave-sms9 krb5kdc[28638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967444, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:24.952849 28651 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:24.953804 28651 webserver.cc:460] Webserver started at http://127.0.0.1:45879/ using document root <none> and password file <none>
I20260620 14:57:24.954342 28651 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:24.954401 28651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:24.954643 28651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:24.956717 28651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "c90668c7ddae44fa94d2855bb22d01b1"
format_stamp: "Formatted at 2026-06-20 14:57:24 on dist-test-slave-sms9"
server_key: "9e8b7bf6e5c5d82482a3cda2917169bd"
server_key_iv: "6954d63c640b13513e8d19cb95640b19"
server_key_version: "encryptionkey@0"
I20260620 14:57:24.957257 28651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "c90668c7ddae44fa94d2855bb22d01b1"
format_stamp: "Formatted at 2026-06-20 14:57:24 on dist-test-slave-sms9"
server_key: "9e8b7bf6e5c5d82482a3cda2917169bd"
server_key_iv: "6954d63c640b13513e8d19cb95640b19"
server_key_version: "encryptionkey@0"
I20260620 14:57:24.960999 28651 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:57:24.963421 28666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:24.964738 28651 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:24.964900 28651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "c90668c7ddae44fa94d2855bb22d01b1"
format_stamp: "Formatted at 2026-06-20 14:57:24 on dist-test-slave-sms9"
server_key: "9e8b7bf6e5c5d82482a3cda2917169bd"
server_key_iv: "6954d63c640b13513e8d19cb95640b19"
server_key_version: "encryptionkey@0"
I20260620 14:57:24.965025 28651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:24.979598 28651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:24.982676 28651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:24.982879 28651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:24.989972 28651 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44127
I20260620 14:57:24.989996 28718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44127 every 8 connection(s)
I20260620 14:57:24.990980 28651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:24.993659 28719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:24.996204 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 28651
I20260620 14:57:24.996349 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:24.996663 7267 external_mini_cluster.cc:1428] Setting key b4a151dccfeff20ea889e788bb5b4397
I20260620 14:57:24.999833 28719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1: Bootstrap starting.
I20260620 14:57:25.001945 28719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:25.002732 28719 log.cc:824] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:25.004631 28719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1: No bootstrap required, opened a new log
I20260620 14:57:25.005726 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:24.998150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:37827 (local address 127.0.0.1:44127)
0620 14:57:24.998661 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:24.998677 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:24.998728 (+ 51us) server_negotiation.cc:407] Connection header received
0620 14:57:24.999386 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:24.999404 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:24.999743 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:25.000082 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:25.000486 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:25.001433 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:25.002403 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:25.002970 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:25.003152 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:25.004119 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:25.004125 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:25.004138 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:25.004513 (+ 375us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:25.004566 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:25.004588 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:25.004739 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:25.004906 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 14:57:25.005049 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":131,"threads_started":1}
I20260620 14:57:25.008658 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:25.008620 28719 raft_consensus.cc:348] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90668c7ddae44fa94d2855bb22d01b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260620 14:57:25.008911 28719 raft_consensus.cc:374] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:25.009003 28719 raft_consensus.cc:729] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90668c7ddae44fa94d2855bb22d01b1, State: Initialized, Role: FOLLOWER
I20260620 14:57:25.009402 28719 consensus_queue.cc:260] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90668c7ddae44fa94d2855bb22d01b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260620 14:57:25.009555 28719 raft_consensus.cc:388] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:25.009642 28719 raft_consensus.cc:482] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:25.009739 28719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:25.010991 28719 raft_consensus.cc:504] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90668c7ddae44fa94d2855bb22d01b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260620 14:57:25.011365 28719 leader_election.cc:302] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c90668c7ddae44fa94d2855bb22d01b1; no voters:
I20260620 14:57:25.011736 28719 leader_election.cc:288] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:25.011839 28729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:25.012127 28729 raft_consensus.cc:686] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [term 1 LEADER]: Becoming Leader. State: Replica: c90668c7ddae44fa94d2855bb22d01b1, State: Running, Role: LEADER
I20260620 14:57:25.012485 28729 consensus_queue.cc:237] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90668c7ddae44fa94d2855bb22d01b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } }
I20260620 14:57:25.013015 28719 sys_catalog.cc:564] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:25.013715 28729 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c90668c7ddae44fa94d2855bb22d01b1. Latest consensus state: current_term: 1 leader_uuid: "c90668c7ddae44fa94d2855bb22d01b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90668c7ddae44fa94d2855bb22d01b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } } }
I20260620 14:57:25.013902 28729 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:25.014313 28738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:25.016060 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:25.011110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:55595 (local address 127.0.0.1:44127)
0620 14:57:25.011335 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:25.011339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:25.011378 (+ 39us) server_negotiation.cc:407] Connection header received
0620 14:57:25.011499 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:25.011502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:25.011551 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:25.011627 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:25.011940 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:25.012697 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:25.013841 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:25.014192 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:25.014291 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:25.015412 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:25.015416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:25.015419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:25.015432 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:25.015459 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:25.015472 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:25.015549 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:25.015898 (+ 349us) server_negotiation.cc:299] Negotiation successful
0620 14:57:25.015933 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260620 14:57:25.016291 28731 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c90668c7ddae44fa94d2855bb22d01b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90668c7ddae44fa94d2855bb22d01b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44127 } } }
I20260620 14:57:25.016511 28731 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:25.018127 28738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:25.024040 28738 catalog_manager.cc:1347] Generated new cluster ID: 7e7a4ed626804b5988125c423910f7c8
I20260620 14:57:25.024119 28738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:25.042305 28738 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:25.043624 28738 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:25.050387 28738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c90668c7ddae44fa94d2855bb22d01b1: Generated new TSK 0
I20260620 14:57:25.051218 28738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:25.067203 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 28651
W20260620 14:57:25.072883 28636 connection.cc:535] client connection to 127.0.0.1:44127 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44127): Connection reset by peer (error 104)
2026-06-20T14:57:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:25 dist-test-slave-sms9 krb5kdc[28750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:25 dist-test-slave-sms9 krb5kdc[28750](info): set up 2 sockets
Jun 20 14:57:25 dist-test-slave-sms9 krb5kdc[28750](info): commencing operation
krb5kdc: starting...
W20260620 14:57:26.122747 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:26 dist-test-slave-sms9 krb5kdc[28750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 14:57:26.246353 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36737
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:26.356200 28763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:26.356532 28763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:26.356607 28763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:26.360693 28763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:26.360785 28763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:26.360821 28763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:26.360878 28763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:26.360929 28763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:26.366232 28763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36737
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:26.367321 28763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:26.368340 28763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:57:26.376964 28763 server_base.cc:1034] running on GCE node
W20260620 14:57:26.376920 28769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:26.376894 28771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:26.376933 28768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:26.377759 28763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:26.378894 28763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:26.380062 28763 hybrid_clock.cc:648] HybridClock initialized: now 1781967446380045 us; error 30 us; skew 500 ppm
Jun 20 14:57:26 dist-test-slave-sms9 krb5kdc[28750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967446, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:26.382704 28763 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 14:57:26.383584 28763 webserver.cc:460] Webserver started at http://127.0.0.1:37707/ using document root <none> and password file <none>
I20260620 14:57:26.384143 28763 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:26.384202 28763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:26.384418 28763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:26.386407 28763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "233e616ceba5489b8f4dc02ab3cbf0d8"
format_stamp: "Formatted at 2026-06-20 14:57:26 on dist-test-slave-sms9"
server_key: "e5714b09022362333f0782682bb601f4"
server_key_iv: "e307e7728fdf53209234022e21b8b441"
server_key_version: "encryptionkey@0"
I20260620 14:57:26.387032 28763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "233e616ceba5489b8f4dc02ab3cbf0d8"
format_stamp: "Formatted at 2026-06-20 14:57:26 on dist-test-slave-sms9"
server_key: "e5714b09022362333f0782682bb601f4"
server_key_iv: "e307e7728fdf53209234022e21b8b441"
server_key_version: "encryptionkey@0"
I20260620 14:57:26.390658 28763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 14:57:26.393064 28778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:26.394279 28763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:57:26.394407 28763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "233e616ceba5489b8f4dc02ab3cbf0d8"
format_stamp: "Formatted at 2026-06-20 14:57:26 on dist-test-slave-sms9"
server_key: "e5714b09022362333f0782682bb601f4"
server_key_iv: "e307e7728fdf53209234022e21b8b441"
server_key_version: "encryptionkey@0"
I20260620 14:57:26.394569 28763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:26.398998 28763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:26.401809 28763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:26.401997 28763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:26.408763 28763 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36737
I20260620 14:57:26.408790 28830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36737 every 8 connection(s)
I20260620 14:57:26.409651 28763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:26.412263 28831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:26.412746 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 28763
I20260620 14:57:26.412904 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:26.413214 7267 external_mini_cluster.cc:1428] Setting key cf5b612328094819152da842019c2bde
I20260620 14:57:26.419178 28834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:26.414742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:46141 (local address 127.0.0.1:36737)
0620 14:57:26.415198 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:26.415219 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:26.415263 (+ 44us) server_negotiation.cc:407] Connection header received
0620 14:57:26.415994 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:26.416017 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:26.416384 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:26.416748 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:26.417740 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:26.417746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:26.417760 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:26.417774 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:26.417845 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:26.417887 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:26.418031 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:26.418097 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 14:57:26.418242 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":129,"threads_started":1}
I20260620 14:57:26.419775 28831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8: Bootstrap starting.
I20260620 14:57:26.422170 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:26.422351 28831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:26.423252 28831 log.cc:824] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:26.425494 28831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8: No bootstrap required, opened a new log
I20260620 14:57:26.426390 28834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:26.424542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:39193 (local address 127.0.0.1:36737)
0620 14:57:26.424675 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:26.424679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:26.424926 (+ 247us) server_negotiation.cc:407] Connection header received
0620 14:57:26.425027 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:26.425029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:26.425057 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:26.425127 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:26.425925 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:26.425928 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:26.425930 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:26.425941 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:26.425964 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:26.425974 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:26.426044 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:26.426161 (+ 117us) server_negotiation.cc:299] Negotiation successful
0620 14:57:26.426196 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 14:57:26.429386 28831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233e616ceba5489b8f4dc02ab3cbf0d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36737 } }
I20260620 14:57:26.429679 28831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:26.429738 28831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e616ceba5489b8f4dc02ab3cbf0d8, State: Initialized, Role: FOLLOWER
I20260620 14:57:26.430130 28831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233e616ceba5489b8f4dc02ab3cbf0d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36737 } }
I20260620 14:57:26.430275 28831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:26.430341 28831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:26.430449 28831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:26.431327 28831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233e616ceba5489b8f4dc02ab3cbf0d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36737 } }
I20260620 14:57:26.431665 28831 leader_election.cc:302] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 233e616ceba5489b8f4dc02ab3cbf0d8; no voters:
I20260620 14:57:26.432008 28831 leader_election.cc:288] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:26.432122 28841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:26.432348 28841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [term 1 LEADER]: Becoming Leader. State: Replica: 233e616ceba5489b8f4dc02ab3cbf0d8, State: Running, Role: LEADER
I20260620 14:57:26.432700 28841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233e616ceba5489b8f4dc02ab3cbf0d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36737 } }
I20260620 14:57:26.433171 28831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:26.434331 28843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 233e616ceba5489b8f4dc02ab3cbf0d8. Latest consensus state: current_term: 1 leader_uuid: "233e616ceba5489b8f4dc02ab3cbf0d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233e616ceba5489b8f4dc02ab3cbf0d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36737 } } }
I20260620 14:57:26.434487 28843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:26.434917 28850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:26.435384 28842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "233e616ceba5489b8f4dc02ab3cbf0d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233e616ceba5489b8f4dc02ab3cbf0d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36737 } } }
I20260620 14:57:26.436013 28842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:26.438067 28850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:26.443835 28850 catalog_manager.cc:1347] Generated new cluster ID: 2ecd14dbc9d84041b180d8ec99789e27
I20260620 14:57:26.443923 28850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:26.459864 28850 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:26.460207 28850 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:26.467042 28850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 233e616ceba5489b8f4dc02ab3cbf0d8: Generated new TSK 0
I20260620 14:57:26.467936 28850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:26.479080 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 28763
2026-06-20T14:57:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:26 dist-test-slave-sms9 krb5kdc[28862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:26 dist-test-slave-sms9 krb5kdc[28862](info): set up 2 sockets
Jun 20 14:57:26 dist-test-slave-sms9 krb5kdc[28862](info): commencing operation
krb5kdc: starting...
W20260620 14:57:27.538641 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:27 dist-test-slave-sms9 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:57:27.665956 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:41209
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:33601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:41209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:27.777441 28875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:27.777740 28875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:27.777782 28875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:27.781782 28875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:27.781850 28875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:27.781870 28875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:27.781888 28875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:27.781903 28875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:27.787333 28875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:33601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:41209
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:41209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:27.788430 28875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:27.789541 28875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:27.798673 28883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:27.798677 28880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:27.798677 28881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:27.799136 28875 server_base.cc:1034] running on GCE node
I20260620 14:57:27.799780 28875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:27.800973 28875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:27.802172 28875 hybrid_clock.cc:648] HybridClock initialized: now 1781967447802141 us; error 44 us; skew 500 ppm
Jun 20 14:57:27 dist-test-slave-sms9 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:27.805250 28875 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:57:27.806229 28875 webserver.cc:460] Webserver started at http://127.7.24.254:38461/ using document root <none> and password file <none>
I20260620 14:57:27.806883 28875 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:27.806951 28875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:27.807124 28875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:27.809221 28875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "a73840f5107445b981b2109f938ccbeb"
format_stamp: "Formatted at 2026-06-20 14:57:27 on dist-test-slave-sms9"
server_key: "3f5fc96b9a9e368849b3e072b1d8c2c2"
server_key_iv: "483bbc71a0b819cd51f282fc690ae6d4"
server_key_version: "encryptionkey@0"
I20260620 14:57:27.809755 28875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "a73840f5107445b981b2109f938ccbeb"
format_stamp: "Formatted at 2026-06-20 14:57:27 on dist-test-slave-sms9"
server_key: "3f5fc96b9a9e368849b3e072b1d8c2c2"
server_key_iv: "483bbc71a0b819cd51f282fc690ae6d4"
server_key_version: "encryptionkey@0"
I20260620 14:57:27.813697 28875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260620 14:57:27.816399 28890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:27.817665 28875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:57:27.817811 28875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "a73840f5107445b981b2109f938ccbeb"
format_stamp: "Formatted at 2026-06-20 14:57:27 on dist-test-slave-sms9"
server_key: "3f5fc96b9a9e368849b3e072b1d8c2c2"
server_key_iv: "483bbc71a0b819cd51f282fc690ae6d4"
server_key_version: "encryptionkey@0"
I20260620 14:57:27.817931 28875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:27.832718 28875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:27.836444 28875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:27.836669 28875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:27.844090 28875 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:41209
I20260620 14:57:27.844110 28942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:41209 every 8 connection(s)
I20260620 14:57:27.845098 28875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:27.848136 28943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:27.852867 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 28875
I20260620 14:57:27.853026 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:27.853334 7267 external_mini_cluster.cc:1428] Setting key 1575e341b0b41ca26399ca589bf2e8e8
I20260620 14:57:27.854341 28943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb: Bootstrap starting.
I20260620 14:57:27.856884 28943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:27.857651 28943 log.cc:824] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:27.859639 28943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb: No bootstrap required, opened a new log
Jun 20 14:57:27 dist-test-slave-sms9 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:27.864027 28943 raft_consensus.cc:348] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73840f5107445b981b2109f938ccbeb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41209 } }
I20260620 14:57:27.864349 28943 raft_consensus.cc:374] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:27.864405 28943 raft_consensus.cc:729] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73840f5107445b981b2109f938ccbeb, State: Initialized, Role: FOLLOWER
I20260620 14:57:27.864996 28943 consensus_queue.cc:260] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73840f5107445b981b2109f938ccbeb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41209 } }
I20260620 14:57:27.865157 28943 raft_consensus.cc:388] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:27.865257 28943 raft_consensus.cc:482] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:27.865392 28943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:27.866370 28943 raft_consensus.cc:504] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73840f5107445b981b2109f938ccbeb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41209 } }
I20260620 14:57:27.866756 28943 leader_election.cc:302] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a73840f5107445b981b2109f938ccbeb; no voters:
I20260620 14:57:27.867149 28943 leader_election.cc:288] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:27.867220 28948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:27.867435 28948 raft_consensus.cc:686] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [term 1 LEADER]: Becoming Leader. State: Replica: a73840f5107445b981b2109f938ccbeb, State: Running, Role: LEADER
I20260620 14:57:27.867779 28948 consensus_queue.cc:237] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73840f5107445b981b2109f938ccbeb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41209 } }
I20260620 14:57:27.868538 28943 sys_catalog.cc:564] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:27.869673 28950 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader a73840f5107445b981b2109f938ccbeb. Latest consensus state: current_term: 1 leader_uuid: "a73840f5107445b981b2109f938ccbeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73840f5107445b981b2109f938ccbeb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41209 } } }
I20260620 14:57:27.869714 28949 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a73840f5107445b981b2109f938ccbeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73840f5107445b981b2109f938ccbeb" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41209 } } }
I20260620 14:57:27.869891 28949 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:27.870198 28950 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:27.870333 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:27.854997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56579 (local address 127.7.24.254:41209)
0620 14:57:27.855475 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:27.855491 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:27.855528 (+ 37us) server_negotiation.cc:407] Connection header received
0620 14:57:27.856207 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:27.856225 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:27.856570 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:27.856919 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:27.857796 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:27.858993 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:27.859922 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:27.860415 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:27.860611 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:27.863624 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:27.863630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:27.863646 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:27.863679 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:27.866102 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:27.866808 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:27.866815 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:27.866824 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:27.867006 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:27.867401 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:27.867405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:27.867407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:27.868049 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:27.868316 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:27.868674 (+ 358us) server_negotiation.cc:299] Negotiation successful
0620 14:57:27.869119 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":154,"threads_started":1}
I20260620 14:57:27.870330 28957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:27.873941 28957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:27.874500 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:27.879992 28957 catalog_manager.cc:1347] Generated new cluster ID: d9fd9157ae4747ee867760a30b88a965
I20260620 14:57:27.880067 28957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:27.887715 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:27.878042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60137 (local address 127.7.24.254:41209)
0620 14:57:27.878443 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:27.878448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:27.878900 (+ 452us) server_negotiation.cc:407] Connection header received
0620 14:57:27.879102 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:27.879105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:27.879155 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:27.879209 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:27.880290 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:27.880878 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:27.881825 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:27.882119 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:27.882247 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:27.883931 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:27.883935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:27.883938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:27.883966 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:27.885644 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:27.886193 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:27.886205 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:27.886207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:27.886284 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:27.886690 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:27.886693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:27.886695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:27.886891 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:27.887103 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:27.887413 (+ 310us) server_negotiation.cc:299] Negotiation successful
0620 14:57:27.887514 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":324}
I20260620 14:57:27.904817 28957 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:27.905889 28957 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:27.909986 28957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a73840f5107445b981b2109f938ccbeb: Generated new TSK 0
I20260620 14:57:27.910777 28957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:27.919065 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 28875
W20260620 14:57:27.924255 28860 connection.cc:535] client connection to 127.7.24.254:41209 recv error: Network error: failed to read from TLS socket (remote: 127.7.24.254:41209): Connection reset by peer (error 104)
2026-06-20T14:57:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:27 dist-test-slave-sms9 krb5kdc[28974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:27 dist-test-slave-sms9 krb5kdc[28974](info): set up 2 sockets
Jun 20 14:57:27 dist-test-slave-sms9 krb5kdc[28974](info): commencing operation
krb5kdc: starting...
W20260620 14:57:28.966735 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:29 dist-test-slave-sms9 krb5kdc[28974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:57:29.094204 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:41177
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:34583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:41177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:29.205623 28987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:29.205977 28987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:29.206070 28987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:29.210263 28987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:29.210371 28987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:29.210417 28987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:29.210493 28987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:29.210551 28987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:29.216348 28987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:34583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:41177
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:41177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:29.217566 28987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:29.218703 28987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:29.227523 28992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:29.227515 28993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:29.227519 28995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:29.227834 28987 server_base.cc:1034] running on GCE node
I20260620 14:57:29.228449 28987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:29.229645 28987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:29.230863 28987 hybrid_clock.cc:648] HybridClock initialized: now 1781967449230844 us; error 46 us; skew 500 ppm
Jun 20 14:57:29 dist-test-slave-sms9 krb5kdc[28974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967449, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:29.233827 28987 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:57:29.235172 28987 webserver.cc:460] Webserver started at http://127.7.24.254:44117/ using document root <none> and password file <none>
I20260620 14:57:29.235901 28987 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:29.236006 28987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:29.236250 28987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:29.238605 28987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "41410d7fea0a49ca8dc38cde6a87f181"
format_stamp: "Formatted at 2026-06-20 14:57:29 on dist-test-slave-sms9"
server_key: "3a6da63472c163837b0854e2ecb82ccd"
server_key_iv: "d44c573fe5d75dc9f5fd5d783b16d4c3"
server_key_version: "encryptionkey@0"
I20260620 14:57:29.239213 28987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "41410d7fea0a49ca8dc38cde6a87f181"
format_stamp: "Formatted at 2026-06-20 14:57:29 on dist-test-slave-sms9"
server_key: "3a6da63472c163837b0854e2ecb82ccd"
server_key_iv: "d44c573fe5d75dc9f5fd5d783b16d4c3"
server_key_version: "encryptionkey@0"
I20260620 14:57:29.243809 28987 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 14:57:29.246604 29002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:29.247866 28987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 14:57:29.248018 28987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "41410d7fea0a49ca8dc38cde6a87f181"
format_stamp: "Formatted at 2026-06-20 14:57:29 on dist-test-slave-sms9"
server_key: "3a6da63472c163837b0854e2ecb82ccd"
server_key_iv: "d44c573fe5d75dc9f5fd5d783b16d4c3"
server_key_version: "encryptionkey@0"
I20260620 14:57:29.248138 28987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:29.266098 28987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:29.269534 28987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:29.269745 28987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:29.276988 28987 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:41177
I20260620 14:57:29.277019 29054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:41177 every 8 connection(s)
I20260620 14:57:29.277951 28987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:29.280977 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 28987
I20260620 14:57:29.281068 29055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:29.281148 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:29.281483 7267 external_mini_cluster.cc:1428] Setting key 10478c1e58eb49a951227ec8c69206e7
I20260620 14:57:29.287842 29055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181: Bootstrap starting.
I20260620 14:57:29.289968 29055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:29.290791 29055 log.cc:824] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:29.291038 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:29.283044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34727 (local address 127.7.24.254:41177)
0620 14:57:29.283553 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:29.283568 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:29.283603 (+ 35us) server_negotiation.cc:407] Connection header received
0620 14:57:29.284542 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:29.284561 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:29.284926 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:29.285273 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:29.285672 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:29.286769 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:29.287637 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:29.288181 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:29.288351 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:29.289272 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:29.289277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:29.289289 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:29.289666 (+ 377us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:29.289716 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:29.289739 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:29.289879 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:29.290053 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 14:57:29.290194 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":168,"threads_started":1}
I20260620 14:57:29.293568 29055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181: No bootstrap required, opened a new log
I20260620 14:57:29.293804 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:29.297884 29055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41410d7fea0a49ca8dc38cde6a87f181" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41177 } }
I20260620 14:57:29.298198 29055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:29.298256 29055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41410d7fea0a49ca8dc38cde6a87f181, State: Initialized, Role: FOLLOWER
I20260620 14:57:29.298746 29055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41410d7fea0a49ca8dc38cde6a87f181" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41177 } }
I20260620 14:57:29.298960 29055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:29.299041 29055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:29.299155 29055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:29.300354 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:29.295862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33431 (local address 127.7.24.254:41177)
0620 14:57:29.296011 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:29.296015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:29.296214 (+ 199us) server_negotiation.cc:407] Connection header received
0620 14:57:29.296347 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:29.296349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:29.296385 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:29.296469 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:29.296816 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:29.297302 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:29.298416 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:29.298737 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:29.298820 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:29.299808 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:29.299811 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:29.299814 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:29.299825 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:29.299850 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:29.299861 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:29.299940 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:29.300149 (+ 209us) server_negotiation.cc:299] Negotiation successful
0620 14:57:29.300210 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260620 14:57:29.300732 29055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41410d7fea0a49ca8dc38cde6a87f181" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41177 } }
I20260620 14:57:29.301106 29055 leader_election.cc:302] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41410d7fea0a49ca8dc38cde6a87f181; no voters:
I20260620 14:57:29.301503 29055 leader_election.cc:288] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:29.301581 29065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:29.301798 29065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [term 1 LEADER]: Becoming Leader. State: Replica: 41410d7fea0a49ca8dc38cde6a87f181, State: Running, Role: LEADER
I20260620 14:57:29.302110 29065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41410d7fea0a49ca8dc38cde6a87f181" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41177 } }
I20260620 14:57:29.302971 29055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:29.304064 29066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41410d7fea0a49ca8dc38cde6a87f181" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41410d7fea0a49ca8dc38cde6a87f181" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41177 } } }
I20260620 14:57:29.304092 29067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41410d7fea0a49ca8dc38cde6a87f181. Latest consensus state: current_term: 1 leader_uuid: "41410d7fea0a49ca8dc38cde6a87f181" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41410d7fea0a49ca8dc38cde6a87f181" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 41177 } } }
I20260620 14:57:29.304327 29066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:29.304350 29067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:29.305343 29074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:29.308305 29074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:29.314254 29074 catalog_manager.cc:1347] Generated new cluster ID: 01dccab1fb814954aebc219d0befa472
I20260620 14:57:29.314338 29074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:29.325630 29074 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:29.327039 29074 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:29.341745 29074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41410d7fea0a49ca8dc38cde6a87f181: Generated new TSK 0
I20260620 14:57:29.342742 29074 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:29.356758 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 28987
W20260620 14:57:29.362643 28972 connection.cc:535] client connection to 127.7.24.254:41177 recv error: Network error: failed to read from TLS socket (remote: 127.7.24.254:41177): Connection reset by peer (error 104)
2026-06-20T14:57:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:29 dist-test-slave-sms9 krb5kdc[29086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:29 dist-test-slave-sms9 krb5kdc[29086](info): set up 2 sockets
Jun 20 14:57:29 dist-test-slave-sms9 krb5kdc[29086](info): commencing operation
krb5kdc: starting...
W20260620 14:57:30.398752 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:30 dist-test-slave-sms9 krb5kdc[29086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:57:30.517952 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:36895
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:36895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:30.628681 29099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:30.628944 29099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:30.628986 29099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:30.632999 29099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:30.633060 29099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:30.633080 29099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:30.633096 29099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:30.633111 29099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:30.639114 29099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:36895
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:36895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:30.640136 29099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:30.641240 29099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:30.650296 29107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:30.650274 29104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:30.650532 29099 server_base.cc:1034] running on GCE node
W20260620 14:57:30.650250 29105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:30.651063 29099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:30.652135 29099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:30.653333 29099 hybrid_clock.cc:648] HybridClock initialized: now 1781967450653316 us; error 33 us; skew 500 ppm
Jun 20 14:57:30 dist-test-slave-sms9 krb5kdc[29086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967450, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:30.659734 29099 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:57:30.660605 29099 webserver.cc:460] Webserver started at http://127.7.24.254:35349/ using document root <none> and password file <none>
I20260620 14:57:30.661123 29099 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:30.661181 29099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:30.661345 29099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:30.663407 29099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "950b105d4b95413e99849830b0e116b9"
format_stamp: "Formatted at 2026-06-20 14:57:30 on dist-test-slave-sms9"
server_key: "0c90d1d976bf16531113a9d6b3ccc57b"
server_key_iv: "781cd0a11e0b9b0587731a11b51e5554"
server_key_version: "encryptionkey@0"
I20260620 14:57:30.664009 29099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "950b105d4b95413e99849830b0e116b9"
format_stamp: "Formatted at 2026-06-20 14:57:30 on dist-test-slave-sms9"
server_key: "0c90d1d976bf16531113a9d6b3ccc57b"
server_key_iv: "781cd0a11e0b9b0587731a11b51e5554"
server_key_version: "encryptionkey@0"
I20260620 14:57:30.667714 29099 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 14:57:30.670212 29114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:30.671473 29099 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:30.671609 29099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "950b105d4b95413e99849830b0e116b9"
format_stamp: "Formatted at 2026-06-20 14:57:30 on dist-test-slave-sms9"
server_key: "0c90d1d976bf16531113a9d6b3ccc57b"
server_key_iv: "781cd0a11e0b9b0587731a11b51e5554"
server_key_version: "encryptionkey@0"
I20260620 14:57:30.671716 29099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:30.676436 29099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:30.679750 29099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:30.679955 29099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:30.687111 29099 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:36895
I20260620 14:57:30.687136 29166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:36895 every 8 connection(s)
I20260620 14:57:30.688074 29099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:30.690868 29167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:30.694630 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 29099
I20260620 14:57:30.694778 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:30.695065 7267 external_mini_cluster.cc:1428] Setting key 26bafbf35c953c793b3983fc99e6ef51
I20260620 14:57:30.696995 29167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9: Bootstrap starting.
I20260620 14:57:30.699594 29167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:30.700474 29167 log.cc:824] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:30.700536 29170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:30.696549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56311 (local address 127.7.24.254:36895)
0620 14:57:30.697000 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:30.697016 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:30.697051 (+ 35us) server_negotiation.cc:407] Connection header received
0620 14:57:30.697733 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:30.697750 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:30.698048 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:30.698340 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:30.699244 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:30.699249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:30.699261 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:30.699270 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:30.699320 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:30.699338 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:30.699432 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:30.699585 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 14:57:30.699770 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":146,"threads_started":1}
I20260620 14:57:30.702713 29167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9: No bootstrap required, opened a new log
I20260620 14:57:30.703697 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:30.706875 29167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950b105d4b95413e99849830b0e116b9" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36895 } }
I20260620 14:57:30.707105 29167 raft_consensus.cc:374] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:30.707183 29167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 950b105d4b95413e99849830b0e116b9, State: Initialized, Role: FOLLOWER
I20260620 14:57:30.707707 29170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:30.705796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44781 (local address 127.7.24.254:36895)
0620 14:57:30.705935 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:30.705940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:30.706164 (+ 224us) server_negotiation.cc:407] Connection header received
0620 14:57:30.706259 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:30.706263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:30.706303 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:30.706408 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:30.707282 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:30.707285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:30.707288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:30.707299 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:30.707327 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:30.707347 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:30.707459 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:30.707542 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 14:57:30.707582 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 14:57:30.707721 29167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950b105d4b95413e99849830b0e116b9" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36895 } }
I20260620 14:57:30.707993 29167 raft_consensus.cc:388] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:30.708068 29167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:30.708166 29167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:30.709282 29167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950b105d4b95413e99849830b0e116b9" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36895 } }
I20260620 14:57:30.709640 29167 leader_election.cc:302] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 950b105d4b95413e99849830b0e116b9; no voters:
I20260620 14:57:30.709986 29167 leader_election.cc:288] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:30.710062 29177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:30.710306 29177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [term 1 LEADER]: Becoming Leader. State: Replica: 950b105d4b95413e99849830b0e116b9, State: Running, Role: LEADER
I20260620 14:57:30.710678 29177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950b105d4b95413e99849830b0e116b9" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36895 } }
I20260620 14:57:30.711345 29167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:30.712464 29178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "950b105d4b95413e99849830b0e116b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950b105d4b95413e99849830b0e116b9" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36895 } } }
I20260620 14:57:30.712474 29179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 950b105d4b95413e99849830b0e116b9. Latest consensus state: current_term: 1 leader_uuid: "950b105d4b95413e99849830b0e116b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "950b105d4b95413e99849830b0e116b9" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 36895 } } }
I20260620 14:57:30.712599 29178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:30.712598 29179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:30.712982 29186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:30.716281 29186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:30.722299 29186 catalog_manager.cc:1347] Generated new cluster ID: 57b13b78cdd5443f9775f43706472400
I20260620 14:57:30.722391 29186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:30.734195 29186 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:30.734587 29186 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:30.741091 29186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 950b105d4b95413e99849830b0e116b9: Generated new TSK 0
I20260620 14:57:30.741799 29186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:30.746932 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 29099
2026-06-20T14:57:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1389 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:30 dist-test-slave-sms9 krb5kdc[29198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:30 dist-test-slave-sms9 krb5kdc[29198](info): set up 2 sockets
Jun 20 14:57:30 dist-test-slave-sms9 krb5kdc[29198](info): commencing operation
krb5kdc: starting...
W20260620 14:57:31.802775 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:31 dist-test-slave-sms9 krb5kdc[29198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:57:31.923585 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:46643
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:46643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:32.054188 29211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:32.054544 29211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:32.054616 29211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:32.058590 29211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:32.058641 29211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:32.058660 29211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:32.058676 29211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:32.058691 29211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:32.064023 29211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:46643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:46643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:32.065109 29211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:32.066159 29211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:32.074663 29219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:32.074716 29216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:32.074716 29217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:32.074913 29211 server_base.cc:1034] running on GCE node
I20260620 14:57:32.075479 29211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:32.076655 29211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:32.077853 29211 hybrid_clock.cc:648] HybridClock initialized: now 1781967452077833 us; error 31 us; skew 500 ppm
Jun 20 14:57:32 dist-test-slave-sms9 krb5kdc[29198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967452, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:32.080776 29211 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:57:32.081712 29211 webserver.cc:460] Webserver started at http://127.7.24.254:40735/ using document root <none> and password file <none>
I20260620 14:57:32.082267 29211 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:32.082329 29211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:32.082599 29211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:32.084692 29211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "04f8317b189940fdbbb7e2a4f02bec79"
format_stamp: "Formatted at 2026-06-20 14:57:32 on dist-test-slave-sms9"
server_key: "81632393b858bec553d3f880a5f352bc"
server_key_iv: "0d3de2652fa3c96ffb04c01d181a4d47"
server_key_version: "encryptionkey@0"
I20260620 14:57:32.085421 29211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "04f8317b189940fdbbb7e2a4f02bec79"
format_stamp: "Formatted at 2026-06-20 14:57:32 on dist-test-slave-sms9"
server_key: "81632393b858bec553d3f880a5f352bc"
server_key_iv: "0d3de2652fa3c96ffb04c01d181a4d47"
server_key_version: "encryptionkey@0"
I20260620 14:57:32.089115 29211 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260620 14:57:32.091583 29226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:32.092787 29211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:57:32.092931 29211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "04f8317b189940fdbbb7e2a4f02bec79"
format_stamp: "Formatted at 2026-06-20 14:57:32 on dist-test-slave-sms9"
server_key: "81632393b858bec553d3f880a5f352bc"
server_key_iv: "0d3de2652fa3c96ffb04c01d181a4d47"
server_key_version: "encryptionkey@0"
I20260620 14:57:32.093042 29211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:32.113267 29211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:32.116626 29211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:32.116882 29211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:32.124403 29211 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:46643
I20260620 14:57:32.124428 29278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:46643 every 8 connection(s)
I20260620 14:57:32.125363 29211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:32.128069 29279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:32.129995 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 29211
I20260620 14:57:32.130139 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:32.130421 7267 external_mini_cluster.cc:1428] Setting key ab4909b9927294ef79f9d2aa8fd97896
I20260620 14:57:32.134562 29279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79: Bootstrap starting.
I20260620 14:57:32.136695 29279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:32.137444 29279 log.cc:824] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79: Log is configured to *not* fsync() on all Append() calls
Jun 20 14:57:32 dist-test-slave-sms9 krb5kdc[29198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:32.139350 29279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79: No bootstrap required, opened a new log
I20260620 14:57:32.142927 29279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f8317b189940fdbbb7e2a4f02bec79" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46643 } }
I20260620 14:57:32.143200 29279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:32.143296 29279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f8317b189940fdbbb7e2a4f02bec79, State: Initialized, Role: FOLLOWER
I20260620 14:57:32.143769 29279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f8317b189940fdbbb7e2a4f02bec79" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46643 } }
I20260620 14:57:32.143924 29279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:32.144013 29279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:32.144104 29282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:32.131880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44947 (local address 127.7.24.254:46643)
0620 14:57:32.132324 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:32.132338 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:32.132372 (+ 34us) server_negotiation.cc:407] Connection header received
0620 14:57:32.133000 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:32.133015 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:32.133327 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:32.133650 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:32.134440 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:32.135502 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:32.136353 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:32.136867 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:32.137035 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:32.139515 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:32.139520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:32.139532 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:32.139557 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:32.141167 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:32.141611 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:32.141621 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:32.141625 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:32.141730 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:32.142042 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:32.142045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:32.142047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:32.142422 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:32.142706 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:32.142940 (+ 234us) server_negotiation.cc:299] Negotiation successful
0620 14:57:32.143263 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":133,"threads_started":1}
I20260620 14:57:32.144110 29279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:32.145323 29279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f8317b189940fdbbb7e2a4f02bec79" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46643 } }
I20260620 14:57:32.145677 29279 leader_election.cc:302] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04f8317b189940fdbbb7e2a4f02bec79; no voters:
I20260620 14:57:32.146028 29279 leader_election.cc:288] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:32.146140 29284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:32.146369 29284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [term 1 LEADER]: Becoming Leader. State: Replica: 04f8317b189940fdbbb7e2a4f02bec79, State: Running, Role: LEADER
I20260620 14:57:32.146736 29284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f8317b189940fdbbb7e2a4f02bec79" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46643 } }
I20260620 14:57:32.147248 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:32.147409 29279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:32.149617 29286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f8317b189940fdbbb7e2a4f02bec79. Latest consensus state: current_term: 1 leader_uuid: "04f8317b189940fdbbb7e2a4f02bec79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f8317b189940fdbbb7e2a4f02bec79" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46643 } } }
I20260620 14:57:32.149730 29286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:32.150506 29285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04f8317b189940fdbbb7e2a4f02bec79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04f8317b189940fdbbb7e2a4f02bec79" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46643 } } }
I20260620 14:57:32.150818 29285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79 [sys.catalog]: This master's current role is: LEADER
W20260620 14:57:32.152261 29303 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:57:32.152340 29303 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:57:32.152451 29305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:32.153319 29305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:32.158525 29282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:32.149377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34339 (local address 127.7.24.254:46643)
0620 14:57:32.149977 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:32.149981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:32.149995 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:57:32.150030 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:32.150035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:32.150080 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:32.150167 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:32.151168 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:32.151946 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:32.152983 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:32.153243 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:32.153357 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:32.154775 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:32.154778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:32.154781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:32.154806 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:32.156592 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:32.157088 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:32.157091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:32.157094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:32.157180 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:32.157503 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:32.157506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:32.157508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:32.157723 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:32.157859 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:32.158191 (+ 332us) server_negotiation.cc:299] Negotiation successful
0620 14:57:32.158314 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535}
I20260620 14:57:32.159611 29305 catalog_manager.cc:1347] Generated new cluster ID: ea58410171dc45779835695eba634546
I20260620 14:57:32.159672 29305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:32.176751 29305 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:32.178005 29305 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:32.189534 29305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04f8317b189940fdbbb7e2a4f02bec79: Generated new TSK 0
I20260620 14:57:32.190380 29305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:32.195582 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 29211
2026-06-20T14:57:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:32 dist-test-slave-sms9 krb5kdc[29310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:32 dist-test-slave-sms9 krb5kdc[29310](info): set up 2 sockets
Jun 20 14:57:32 dist-test-slave-sms9 krb5kdc[29310](info): commencing operation
krb5kdc: starting...
W20260620 14:57:33.238867 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:33 dist-test-slave-sms9 krb5kdc[29310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:57:33.365414 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:42111
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:40045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:42111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:33.473906 29323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:33.474161 29323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:33.474203 29323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:33.478085 29323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:33.478142 29323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:33.478160 29323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:33.478176 29323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:33.478191 29323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:33.483297 29323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:40045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:42111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:42111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:33.484210 29323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:33.485127 29323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:33.493376 29328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:33.493391 29329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:33.493439 29323 server_base.cc:1034] running on GCE node
W20260620 14:57:33.493417 29331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:33.494146 29323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:33.495332 29323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:33.496547 29323 hybrid_clock.cc:648] HybridClock initialized: now 1781967453496528 us; error 55 us; skew 500 ppm
Jun 20 14:57:33 dist-test-slave-sms9 krb5kdc[29310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967453, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:33.499851 29323 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:57:33.501014 29323 webserver.cc:460] Webserver started at http://127.7.24.254:37581/ using document root <none> and password file <none>
I20260620 14:57:33.501827 29323 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:33.501909 29323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:33.502162 29323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:33.504204 29323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "f46881442f364f57b25a6851c871ebfd"
format_stamp: "Formatted at 2026-06-20 14:57:33 on dist-test-slave-sms9"
server_key: "e9f03d8546dd36f6a78dbb9c4941f754"
server_key_iv: "46209f662f6da53a0eb48dea7ccaf072"
server_key_version: "encryptionkey@0"
I20260620 14:57:33.504765 29323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "f46881442f364f57b25a6851c871ebfd"
format_stamp: "Formatted at 2026-06-20 14:57:33 on dist-test-slave-sms9"
server_key: "e9f03d8546dd36f6a78dbb9c4941f754"
server_key_iv: "46209f662f6da53a0eb48dea7ccaf072"
server_key_version: "encryptionkey@0"
I20260620 14:57:33.508432 29323 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 14:57:33.510752 29338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:33.511914 29323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260620 14:57:33.512071 29323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "f46881442f364f57b25a6851c871ebfd"
format_stamp: "Formatted at 2026-06-20 14:57:33 on dist-test-slave-sms9"
server_key: "e9f03d8546dd36f6a78dbb9c4941f754"
server_key_iv: "46209f662f6da53a0eb48dea7ccaf072"
server_key_version: "encryptionkey@0"
I20260620 14:57:33.512188 29323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:33.526695 29323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:33.529407 29323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:33.529600 29323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:33.536506 29323 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:42111
I20260620 14:57:33.536583 29390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:42111 every 8 connection(s)
I20260620 14:57:33.537441 29323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:33.540150 29391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:33.541451 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 29323
I20260620 14:57:33.541587 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:33.541877 7267 external_mini_cluster.cc:1428] Setting key c3da17af6cf71cdc8da791b6636bdd7e
I20260620 14:57:33.546137 29391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd: Bootstrap starting.
I20260620 14:57:33.548354 29391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:33.549235 29391 log.cc:824] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:33.550599 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:33.543171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34253 (local address 127.7.24.254:42111)
0620 14:57:33.543560 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:33.543574 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:33.543608 (+ 34us) server_negotiation.cc:407] Connection header received
0620 14:57:33.544220 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:33.544236 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:33.544563 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:33.544825 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:33.545185 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:33.546144 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:33.547107 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:33.547568 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:33.547746 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:33.548637 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:33.548645 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:33.548663 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:33.549147 (+ 484us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:33.549211 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:33.549243 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:33.549351 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:33.549566 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 14:57:33.549779 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":127,"threads_started":1}
I20260620 14:57:33.551262 29391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd: No bootstrap required, opened a new log
I20260620 14:57:33.553263 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:33.555395 29391 raft_consensus.cc:348] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46881442f364f57b25a6851c871ebfd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42111 } }
I20260620 14:57:33.555662 29391 raft_consensus.cc:374] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:33.555742 29391 raft_consensus.cc:729] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46881442f364f57b25a6851c871ebfd, State: Initialized, Role: FOLLOWER
I20260620 14:57:33.556232 29391 consensus_queue.cc:260] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46881442f364f57b25a6851c871ebfd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42111 } }
I20260620 14:57:33.556389 29391 raft_consensus.cc:388] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:33.556471 29391 raft_consensus.cc:482] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:33.556573 29391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:33.557601 29391 raft_consensus.cc:504] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46881442f364f57b25a6851c871ebfd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42111 } }
I20260620 14:57:33.557925 29391 leader_election.cc:302] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f46881442f364f57b25a6851c871ebfd; no voters:
I20260620 14:57:33.558225 29391 leader_election.cc:288] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:33.558336 29401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:33.558588 29401 raft_consensus.cc:686] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [term 1 LEADER]: Becoming Leader. State: Replica: f46881442f364f57b25a6851c871ebfd, State: Running, Role: LEADER
I20260620 14:57:33.558913 29401 consensus_queue.cc:237] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46881442f364f57b25a6851c871ebfd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42111 } }
I20260620 14:57:33.559558 29391 sys_catalog.cc:564] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:33.559952 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:33.555553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52961 (local address 127.7.24.254:42111)
0620 14:57:33.555691 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:33.555695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:33.555923 (+ 228us) server_negotiation.cc:407] Connection header received
0620 14:57:33.556063 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:33.556066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:33.556111 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:33.556195 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:33.556467 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:33.557206 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:33.558085 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:33.558487 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:33.558586 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:33.559476 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:33.559480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:33.559482 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:33.559496 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:33.559524 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:33.559537 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:33.559615 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:33.559756 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 14:57:33.559806 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 14:57:33.560096 29401 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f46881442f364f57b25a6851c871ebfd. Latest consensus state: current_term: 1 leader_uuid: "f46881442f364f57b25a6851c871ebfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46881442f364f57b25a6851c871ebfd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42111 } } }
I20260620 14:57:33.560206 29401 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:33.560726 29402 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f46881442f364f57b25a6851c871ebfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46881442f364f57b25a6851c871ebfd" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 42111 } } }
I20260620 14:57:33.560820 29402 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:33.561103 29410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:33.564096 29410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:33.569692 29410 catalog_manager.cc:1347] Generated new cluster ID: 138526a6d1f8421fa85086ec5758fdd4
I20260620 14:57:33.569773 29410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:33.595001 29410 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:33.596072 29410 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:33.603456 29410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f46881442f364f57b25a6851c871ebfd: Generated new TSK 0
I20260620 14:57:33.604487 29410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:33.617362 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 29323
2026-06-20T14:57:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:33 dist-test-slave-sms9 krb5kdc[29422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:33 dist-test-slave-sms9 krb5kdc[29422](info): set up 2 sockets
Jun 20 14:57:33 dist-test-slave-sms9 krb5kdc[29422](info): commencing operation
krb5kdc: starting...
W20260620 14:57:34.654940 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:34 dist-test-slave-sms9 krb5kdc[29422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:57:34.779567 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:33195
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:33195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:34.889375 29435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:34.889664 29435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:34.889706 29435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:34.893829 29435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:34.893909 29435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:34.893929 29435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:34.893946 29435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:34.893961 29435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:34.899358 29435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:33195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:33195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:34.900480 29435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:34.901539 29435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:34.910063 29441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:34.910141 29440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:34.910077 29443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:34.910156 29435 server_base.cc:1034] running on GCE node
I20260620 14:57:34.910908 29435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:34.912009 29435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:34.913172 29435 hybrid_clock.cc:648] HybridClock initialized: now 1781967454913159 us; error 33 us; skew 500 ppm
Jun 20 14:57:34 dist-test-slave-sms9 krb5kdc[29422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967454, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:34.916065 29435 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:57:34.917021 29435 webserver.cc:460] Webserver started at http://127.7.24.254:41563/ using document root <none> and password file <none>
I20260620 14:57:34.917598 29435 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:34.917690 29435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:34.917925 29435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:34.920081 29435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "43f00a0c9bca4b09aabbfc77986afe8f"
format_stamp: "Formatted at 2026-06-20 14:57:34 on dist-test-slave-sms9"
server_key: "dcd0a2498523923cf913a39df7582867"
server_key_iv: "f10e76ab5d87cae3b73c155af61b2d0f"
server_key_version: "encryptionkey@0"
I20260620 14:57:34.920674 29435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "43f00a0c9bca4b09aabbfc77986afe8f"
format_stamp: "Formatted at 2026-06-20 14:57:34 on dist-test-slave-sms9"
server_key: "dcd0a2498523923cf913a39df7582867"
server_key_iv: "f10e76ab5d87cae3b73c155af61b2d0f"
server_key_version: "encryptionkey@0"
I20260620 14:57:34.924528 29435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260620 14:57:34.927080 29450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:34.928368 29435 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:34.928521 29435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "43f00a0c9bca4b09aabbfc77986afe8f"
format_stamp: "Formatted at 2026-06-20 14:57:34 on dist-test-slave-sms9"
server_key: "dcd0a2498523923cf913a39df7582867"
server_key_iv: "f10e76ab5d87cae3b73c155af61b2d0f"
server_key_version: "encryptionkey@0"
I20260620 14:57:34.928638 29435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:34.933459 29435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:34.937122 29435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:34.937345 29435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:34.944646 29435 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:33195
I20260620 14:57:34.944665 29502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:33195 every 8 connection(s)
I20260620 14:57:34.945585 29435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:34.945832 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 29435
I20260620 14:57:34.945966 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:34.946305 7267 external_mini_cluster.cc:1428] Setting key f6fa8863af09b816d33989b7dd72024d
I20260620 14:57:34.948781 29503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:34.951719 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:34.947756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45003 (local address 127.7.24.254:33195)
0620 14:57:34.948224 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:34.948238 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:34.948272 (+ 34us) server_negotiation.cc:407] Connection header received
0620 14:57:34.949012 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:34.949029 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:34.949298 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:34.949642 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:34.950452 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:34.950496 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:34.950511 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:34.950522 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:34.950584 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:34.950604 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:34.950758 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:34.950827 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 14:57:34.950971 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":145,"threads_started":1}
I20260620 14:57:34.954566 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:34.955104 29503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f: Bootstrap starting.
I20260620 14:57:34.957696 29503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:34.958652 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:34.956671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55287 (local address 127.7.24.254:33195)
0620 14:57:34.956871 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:34.956878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:34.957138 (+ 260us) server_negotiation.cc:407] Connection header received
0620 14:57:34.957315 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:34.957344 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:34.957382 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:34.957515 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:34.958243 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:34.958247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:34.958249 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 14:57:34.958262 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:34.958291 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 14:57:34.958304 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:34.958392 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:34.958490 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 14:57:34.958534 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260620 14:57:34.958645 29503 log.cc:824] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:34.960561 29503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f: No bootstrap required, opened a new log
I20260620 14:57:34.964305 29503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f00a0c9bca4b09aabbfc77986afe8f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33195 } }
I20260620 14:57:34.964529 29503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 14:57:34.964607 29503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43f00a0c9bca4b09aabbfc77986afe8f, State: Initialized, Role: FOLLOWER
I20260620 14:57:34.965046 29503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f00a0c9bca4b09aabbfc77986afe8f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33195 } }
I20260620 14:57:34.965168 29503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 14:57:34.965233 29503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 14:57:34.965361 29503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:34.966266 29503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f00a0c9bca4b09aabbfc77986afe8f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33195 } }
I20260620 14:57:34.966674 29503 leader_election.cc:302] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43f00a0c9bca4b09aabbfc77986afe8f; no voters:
I20260620 14:57:34.967020 29503 leader_election.cc:288] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 14:57:34.967140 29513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:34.967403 29513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [term 1 LEADER]: Becoming Leader. State: Replica: 43f00a0c9bca4b09aabbfc77986afe8f, State: Running, Role: LEADER
I20260620 14:57:34.967730 29513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f00a0c9bca4b09aabbfc77986afe8f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33195 } }
I20260620 14:57:34.968376 29503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:34.969393 29515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43f00a0c9bca4b09aabbfc77986afe8f. Latest consensus state: current_term: 1 leader_uuid: "43f00a0c9bca4b09aabbfc77986afe8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f00a0c9bca4b09aabbfc77986afe8f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33195 } } }
I20260620 14:57:34.969516 29515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:34.969849 29514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43f00a0c9bca4b09aabbfc77986afe8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f00a0c9bca4b09aabbfc77986afe8f" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 33195 } } }
I20260620 14:57:34.969918 29522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:34.973150 29522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:34.969949 29514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:34.979251 29522 catalog_manager.cc:1347] Generated new cluster ID: 4bc877b43ea046529eaccda4d1daa97c
I20260620 14:57:34.979346 29522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:34.989141 29522 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:34.989467 29522 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:34.995850 29522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43f00a0c9bca4b09aabbfc77986afe8f: Generated new TSK 0
I20260620 14:57:34.996603 29522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 14:57:35.000571 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 29435
2026-06-20T14:57:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1383 ms)
[----------] 18 tests from AuthTokenIssuingTest (25623 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:35 dist-test-slave-sms9 krb5kdc[29534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:35 dist-test-slave-sms9 krb5kdc[29534](info): set up 2 sockets
Jun 20 14:57:35 dist-test-slave-sms9 krb5kdc[29534](info): commencing operation
krb5kdc: starting...
W20260620 14:57:36.042761 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:57:36.169525 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:46587
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:46587,127.7.24.253:45493,127.7.24.252:43901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:36.283149 29547 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 14:57:36.283496 29547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:36.283568 29547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:36.283651 29547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:36.287752 29547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:36.287844 29547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:36.287881 29547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:36.287937 29547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:36.287976 29547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:36.293689 29547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:46587,127.7.24.253:45493,127.7.24.252:43901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:46587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:36.294909 29547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:36.295967 29547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:36.304530 29555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:36.304756 29552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:36.304845 29553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:36.304997 29547 server_base.cc:1034] running on GCE node
I20260620 14:57:36.305562 29547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:36.306777 29547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:36.307960 29547 hybrid_clock.cc:648] HybridClock initialized: now 1781967456307942 us; error 36 us; skew 500 ppm
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:36.315598 29547 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:57:36.316670 29547 webserver.cc:460] Webserver started at http://127.7.24.254:37695/ using document root <none> and password file <none>
I20260620 14:57:36.317268 29547 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:36.317373 29547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:36.317619 29547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:36.319789 29547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "b005e650e15d434cad5a0e3b89278369"
format_stamp: "Formatted at 2026-06-20 14:57:36 on dist-test-slave-sms9"
server_key: "440b2ffdeb7a383a72f406f7ea8c9569"
server_key_iv: "d2fea4ba5f50a99189843e696b418363"
server_key_version: "encryptionkey@0"
I20260620 14:57:36.320431 29547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "b005e650e15d434cad5a0e3b89278369"
format_stamp: "Formatted at 2026-06-20 14:57:36 on dist-test-slave-sms9"
server_key: "440b2ffdeb7a383a72f406f7ea8c9569"
server_key_iv: "d2fea4ba5f50a99189843e696b418363"
server_key_version: "encryptionkey@0"
I20260620 14:57:36.324427 29547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 14:57:36.327032 29562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:36.328339 29547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:57:36.328519 29547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "b005e650e15d434cad5a0e3b89278369"
format_stamp: "Formatted at 2026-06-20 14:57:36 on dist-test-slave-sms9"
server_key: "440b2ffdeb7a383a72f406f7ea8c9569"
server_key_iv: "d2fea4ba5f50a99189843e696b418363"
server_key_version: "encryptionkey@0"
I20260620 14:57:36.328645 29547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:36.340008 29547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:36.343197 29547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:36.343427 29547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:36.350584 29547 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:46587
I20260620 14:57:36.350903 29614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:46587 every 8 connection(s)
I20260620 14:57:36.351615 29547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:36.354357 29615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:36.356590 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 29547
I20260620 14:57:36.356729 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:36.357033 7267 external_mini_cluster.cc:1428] Setting key 6e2105d7c150121058de2cddc0a6bf43
I20260620 14:57:36.359169 29615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.24.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.253@KRBTEST.COM" created.
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:36.374852 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.362644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48734 (local address 127.7.24.254:46587)
0620 14:57:36.362927 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.362938 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.362985 (+ 47us) server_negotiation.cc:407] Connection header received
0620 14:57:36.363419 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.363423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.363594 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.363776 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.364609 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.365540 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.366596 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.367073 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.367243 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.369861 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.369870 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.369887 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.369917 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.371638 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.372278 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.372282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.372283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.372407 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.372780 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.372781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.372783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.373136 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.373236 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.373762 (+ 526us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.373949 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":87,"threads_started":1}
I20260620 14:57:36.374868 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.361605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:46587 (local address 127.0.0.1:48734)
0620 14:57:36.362124 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.362166 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:36.363112 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:36.363772 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:36.363777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:36.364211 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:36.364463 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.364476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.365676 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.365679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.366432 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.366441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.367225 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.367229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.367378 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.368044 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:36.368070 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:36.369697 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:36.371788 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.371803 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.371818 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.372164 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.372511 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.372516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.372518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.372683 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.373259 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:36.373267 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:36.373459 (+ 192us) client_negotiation.cc:769] Sending connection context
0620 14:57:36.373669 (+ 210us) client_negotiation.cc:240] Negotiation successful
0620 14:57:36.374027 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":10,"thread_start_us":149,"threads_started":1}
I20260620 14:57:36.377549 29615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:57:36.378297 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.378045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:45493 (local address 127.0.0.1:35506)
0620 14:57:36.378163 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.378234 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:45493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 14:57:36.378584 29563 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.24.253:45493: connect: Connection refused (error 111)
I20260620 14:57:36.379422 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.379198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:45493 (local address 127.0.0.1:35508)
0620 14:57:36.379300 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.379349 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:45493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260620 14:57:36.379586 29615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.253:45493: Network error: Client connection negotiation failed: client connection to 127.7.24.253:45493: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab.
Entry for principal kudu/127.7.24.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab.
I20260620 14:57:36.386289 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.253:45493
--webserver_interface=127.7.24.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab
--principal=kudu/127.7.24.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:46587,127.7.24.253:45493,127.7.24.252:43901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:57:36.428812 29615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } attempt: 1
I20260620 14:57:36.429755 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.429371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:45493 (local address 127.0.0.1:35520)
0620 14:57:36.429567 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.429648 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:45493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260620 14:57:36.430555 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.430282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:45493 (local address 127.0.0.1:35522)
0620 14:57:36.430394 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.430448 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:45493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260620 14:57:36.430699 29615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.253:45493: Network error: Client connection negotiation failed: client connection to 127.7.24.253:45493: connect: Connection refused (error 111)
I20260620 14:57:36.498869 29615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } attempt: 2
I20260620 14:57:36.499740 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.499412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:45493 (local address 127.0.0.1:35532)
0620 14:57:36.499582 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.499658 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:45493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260620 14:57:36.500535 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.500283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:45493 (local address 127.0.0.1:35546)
0620 14:57:36.500399 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.500445 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:45493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260620 14:57:36.500752 29615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.253:45493: Network error: Client connection negotiation failed: client connection to 127.7.24.253:45493: connect: Connection refused (error 111)
W20260620 14:57:36.501468 29622 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 14:57:36.501775 29622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:36.501834 29622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:36.501928 29622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:36.506121 29622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:36.506217 29622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:36.506264 29622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:36.506322 29622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:36.506361 29622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:36.511924 29622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:46587,127.7.24.253:45493,127.7.24.252:43901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab
--principal=kudu/127.7.24.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.253:45493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.24.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:36.513077 29622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:36.514114 29622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:36.522826 29628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:36.522882 29630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:36.522850 29627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:36.523018 29622 server_base.cc:1034] running on GCE node
I20260620 14:57:36.523736 29622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:36.524832 29622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:36.526058 29622 hybrid_clock.cc:648] HybridClock initialized: now 1781967456526042 us; error 24 us; skew 500 ppm
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:36.529386 29622 init.cc:377] Logged in from keytab as kudu/127.7.24.253@KRBTEST.COM (short username kudu)
I20260620 14:57:36.530362 29622 webserver.cc:460] Webserver started at http://127.7.24.253:40923/ using document root <none> and password file <none>
I20260620 14:57:36.531028 29622 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:36.531136 29622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:36.531396 29622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:36.533922 29622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/data/instance:
uuid: "2852623339b8488e9365dfb1a4c36abc"
format_stamp: "Formatted at 2026-06-20 14:57:36 on dist-test-slave-sms9"
server_key: "9cf2979ba45fce47016d21133d7f5207"
server_key_iv: "aefbfcca00bcc0ee2ee7530447ccb0e8"
server_key_version: "encryptionkey@0"
I20260620 14:57:36.534675 29622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/wal/instance:
uuid: "2852623339b8488e9365dfb1a4c36abc"
format_stamp: "Formatted at 2026-06-20 14:57:36 on dist-test-slave-sms9"
server_key: "9cf2979ba45fce47016d21133d7f5207"
server_key_iv: "aefbfcca00bcc0ee2ee7530447ccb0e8"
server_key_version: "encryptionkey@0"
I20260620 14:57:36.538959 29622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 14:57:36.541565 29637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:36.542954 29622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:57:36.543108 29622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/wal
uuid: "2852623339b8488e9365dfb1a4c36abc"
format_stamp: "Formatted at 2026-06-20 14:57:36 on dist-test-slave-sms9"
server_key: "9cf2979ba45fce47016d21133d7f5207"
server_key_iv: "aefbfcca00bcc0ee2ee7530447ccb0e8"
server_key_version: "encryptionkey@0"
I20260620 14:57:36.543227 29622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:36.574156 29622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:36.577601 29622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:36.577826 29622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:36.584797 29622 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.253:45493
I20260620 14:57:36.585106 29689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.253:45493 every 8 connection(s)
I20260620 14:57:36.585808 29622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
I20260620 14:57:36.588680 29690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:36.591922 29615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } attempt: 3
I20260620 14:57:36.593226 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 29622
I20260620 14:57:36.593364 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-1/wal/instance
I20260620 14:57:36.592391 29690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:57:36.593684 7267 external_mini_cluster.cc:1428] Setting key b6d8bdb18e75e46d2b470b391755782d
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.24.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.252@KRBTEST.COM" created.
I20260620 14:57:36.606156 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.592513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:45493 (local address 127.0.0.1:35558)
0620 14:57:36.592685 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.592709 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:36.592794 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:36.595645 (+ 2851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:36.595651 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:36.595907 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:36.596075 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.596082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.598086 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.598092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.598691 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.598700 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.599506 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.599513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.599667 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.600309 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:36.600327 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:36.601724 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:36.604074 (+ 2350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.604077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.604080 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.604404 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.604734 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.604740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.604741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.604827 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.605630 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:36.605633 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:36.605743 (+ 110us) client_negotiation.cc:769] Sending connection context
0620 14:57:36.605848 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 14:57:36.605991 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260620 14:57:36.607372 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.595123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.7.24.254:46587)
0620 14:57:36.595273 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.595278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.595614 (+ 336us) server_negotiation.cc:407] Connection header received
0620 14:57:36.595840 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.595843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.595895 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.595996 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.596985 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.597737 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.599355 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.600317 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.600459 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.603142 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.603149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.603152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.603179 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.604825 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.605387 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.605389 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.605396 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.605462 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.605801 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.605822 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.605827 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.606611 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.606711 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.607126 (+ 415us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.607238 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 14:57:36.607363 29692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.593122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35558 (local address 127.7.24.253:45493)
0620 14:57:36.593564 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.593584 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.593630 (+ 46us) server_negotiation.cc:407] Connection header received
0620 14:57:36.594316 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.594332 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.595310 (+ 978us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.596252 (+ 942us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.596639 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.597969 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.598816 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.599360 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.599513 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.601883 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.601889 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.601900 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.601922 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.603948 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.604522 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.604527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.604531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.604633 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.604976 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.604980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.604981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.605542 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.605679 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.606018 (+ 339us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.606310 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":295,"spinlock_wait_cycles":39168,"thread_start_us":150,"threads_started":1}
I20260620 14:57:36.608502 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.595057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:46587 (local address 127.0.0.1:48746)
0620 14:57:36.595497 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.595543 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:36.595692 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:36.596046 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:36.596055 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:36.596390 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:36.596836 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.596855 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.597869 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.597876 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.599219 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.599234 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.600499 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.600503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.600615 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.601114 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:36.601139 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:36.603002 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:36.604958 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.604966 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.604973 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.605284 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.605560 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.605564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.605566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.605655 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.606736 (+ 1081us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:36.606745 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:36.606880 (+ 135us) client_negotiation.cc:769] Sending connection context
0620 14:57:36.607037 (+ 157us) client_negotiation.cc:240] Negotiation successful
0620 14:57:36.607236 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":1,"thread_start_us":140,"threads_started":1}
I20260620 14:57:36.609635 29615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:57:36.609613 29690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:57:36.610364 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.610083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36128)
0620 14:57:36.610204 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.610280 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 14:57:36.611099 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.610819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36132)
0620 14:57:36.610965 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.611021 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260620 14:57:36.611315 29615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:43901: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
I20260620 14:57:36.620520 29692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.610631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35564 (local address 127.7.24.253:45493)
0620 14:57:36.610949 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.610952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.610964 (+ 12us) server_negotiation.cc:407] Connection header received
0620 14:57:36.610991 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.610993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.611024 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.611119 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.611562 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.612042 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.612769 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.613001 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.613107 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.617355 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.617359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.617361 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.617391 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.618844 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.619267 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.619270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.619271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.619341 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.619714 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.619716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.619717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.619882 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.619971 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.620295 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.620399 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":30}
I20260620 14:57:36.620613 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.610403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:45493 (local address 127.0.0.1:35564)
0620 14:57:36.610648 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.610662 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:36.610743 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:36.611127 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:36.611130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:36.611326 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:36.611454 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.611461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.612154 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.612157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.612661 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.612668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.613104 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.613107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.613248 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.613720 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:36.613734 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:36.617219 (+ 3485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:36.618971 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.618976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.618979 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.619177 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.619452 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.619485 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.619490 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.619607 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.620006 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:36.620012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:36.620122 (+ 110us) client_negotiation.cc:769] Sending connection context
0620 14:57:36.620255 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 14:57:36.620478 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":84}
I20260620 14:57:36.621454 29690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:57:36.622225 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.621914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36144)
0620 14:57:36.622019 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.622153 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260620 14:57:36.622542 29641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab.
Entry for principal kudu/127.7.24.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab.
I20260620 14:57:36.623497 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.623278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36154)
0620 14:57:36.623398 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.623449 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260620 14:57:36.623649 29690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:43901: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
I20260620 14:57:36.623764 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.252:43901
--webserver_interface=127.7.24.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab
--principal=kudu/127.7.24.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:43559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:46587,127.7.24.253:45493,127.7.24.252:43901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:57:36.642529 29615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } attempt: 1
I20260620 14:57:36.643368 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.643076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36160)
0620 14:57:36.643224 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.643293 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 14:57:36.644112 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.643884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36164)
0620 14:57:36.644004 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.644049 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260620 14:57:36.644263 29615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:43901: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
I20260620 14:57:36.673046 29690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } attempt: 1
I20260620 14:57:36.674124 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.673813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36174)
0620 14:57:36.673980 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.674044 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260620 14:57:36.674985 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.674767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36186)
0620 14:57:36.674870 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.674920 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260620 14:57:36.675231 29690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:43901: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
I20260620 14:57:36.719437 29615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } attempt: 2
I20260620 14:57:36.720321 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.720031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36198)
0620 14:57:36.720173 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.720245 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260620 14:57:36.721022 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.720790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36200)
0620 14:57:36.720901 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.720951 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 14:57:36.721179 29615 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:43901: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
W20260620 14:57:36.737816 29697 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 14:57:36.738144 29697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:36.738168 29697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:36.738252 29697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:36.742555 29697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:36.742625 29697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:36.742646 29697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:36.742664 29697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:36.742717 29697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:36.743430 29690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } attempt: 2
I20260620 14:57:36.744280 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.743983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36202)
0620 14:57:36.744131 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.744206 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260620 14:57:36.745019 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.744808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36216)
0620 14:57:36.744911 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.744959 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260620 14:57:36.745172 29690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:43901: Network error: Client connection negotiation failed: client connection to 127.7.24.252:43901: connect: Connection refused (error 111)
I20260620 14:57:36.748713 29697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:43559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:46587,127.7.24.253:45493,127.7.24.252:43901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab
--principal=kudu/127.7.24.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.252:43901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.24.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:36.749794 29697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:36.750892 29697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:36.759477 29703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:36.759505 29702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:36.759477 29705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:36.759759 29697 server_base.cc:1034] running on GCE node
I20260620 14:57:36.760417 29697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:36.761533 29697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:36.762715 29697 hybrid_clock.cc:648] HybridClock initialized: now 1781967456762694 us; error 42 us; skew 500 ppm
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:36.765486 29697 init.cc:377] Logged in from keytab as kudu/127.7.24.252@KRBTEST.COM (short username kudu)
I20260620 14:57:36.766408 29697 webserver.cc:460] Webserver started at http://127.7.24.252:36495/ using document root <none> and password file <none>
I20260620 14:57:36.766994 29697 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:36.767086 29697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:36.767314 29697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:36.769364 29697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/data/instance:
uuid: "747905212037498996bd535b1190aa40"
format_stamp: "Formatted at 2026-06-20 14:57:36 on dist-test-slave-sms9"
server_key: "e3009f3a987b79b1528c7f11c5bc5247"
server_key_iv: "a0088c16deca5b471541419a93eedd4f"
server_key_version: "encryptionkey@0"
I20260620 14:57:36.769976 29697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/wal/instance:
uuid: "747905212037498996bd535b1190aa40"
format_stamp: "Formatted at 2026-06-20 14:57:36 on dist-test-slave-sms9"
server_key: "e3009f3a987b79b1528c7f11c5bc5247"
server_key_iv: "a0088c16deca5b471541419a93eedd4f"
server_key_version: "encryptionkey@0"
I20260620 14:57:36.773663 29697 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260620 14:57:36.776137 29712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:36.777423 29697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 14:57:36.777570 29697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/wal
uuid: "747905212037498996bd535b1190aa40"
format_stamp: "Formatted at 2026-06-20 14:57:36 on dist-test-slave-sms9"
server_key: "e3009f3a987b79b1528c7f11c5bc5247"
server_key_iv: "a0088c16deca5b471541419a93eedd4f"
server_key_version: "encryptionkey@0"
I20260620 14:57:36.777683 29697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:36.795828 29697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:36.798736 29697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:36.798946 29697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:36.806145 29697 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.252:43901
I20260620 14:57:36.806449 29764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.252:43901 every 8 connection(s)
I20260620 14:57:36.807212 29697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
I20260620 14:57:36.809962 29765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:36.810261 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 29697
I20260620 14:57:36.810424 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781967176660325-7267-0/minicluster-data/master-2/wal/instance
I20260620 14:57:36.810730 7267 external_mini_cluster.cc:1428] Setting key c92ab510b251539b78a6553bef96786d
I20260620 14:57:36.815079 29765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:57:36.820328 29615 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } attempt: 3
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
I20260620 14:57:36.827081 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.812085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36222 (local address 127.7.24.252:43901)
0620 14:57:36.812499 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.812517 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.812560 (+ 43us) server_negotiation.cc:407] Connection header received
0620 14:57:36.813409 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.813433 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.813857 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.814224 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.814983 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.816457 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.817637 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.818324 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.818517 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.821427 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.821436 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.821452 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.821489 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.823487 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.823982 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.823989 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.824004 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.824142 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.824505 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.824510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.824512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.825043 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.825271 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.825580 (+ 309us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.825884 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":110,"threads_started":1}
I20260620 14:57:36.827651 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.817094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.7.24.254:46587)
0620 14:57:36.817226 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.817231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.817560 (+ 329us) server_negotiation.cc:407] Connection header received
0620 14:57:36.817781 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.817784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.817831 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.817908 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.818783 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.819515 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.820645 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.820999 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.821097 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.823674 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.823678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.823680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.823708 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.825666 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.826259 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.826265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.826267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.826344 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.826676 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.826681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.826683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.826931 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.827024 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.827450 (+ 426us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.827546 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:57:36.828269 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.817009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:46587 (local address 127.0.0.1:48750)
0620 14:57:36.817437 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.817477 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:36.817615 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:36.817980 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:36.817986 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:36.818485 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:36.818640 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.818649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.819660 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.819662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.820521 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.820533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.821157 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.821163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.821266 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.821767 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:36.821781 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:36.823568 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:36.825816 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.825825 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.825831 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.826144 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.826447 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.826449 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.826451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.826584 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.827080 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:36.827085 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:36.827212 (+ 127us) client_negotiation.cc:769] Sending connection context
0620 14:57:36.827849 (+ 637us) client_negotiation.cc:240] Negotiation successful
0620 14:57:36.828037 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":129,"threads_started":1}
I20260620 14:57:36.829370 29765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:57:36.830992 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.820743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36228)
0620 14:57:36.820920 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.820934 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:36.821002 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:36.821459 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:36.821462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:36.821922 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:36.822033 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.822040 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.822941 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.822944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.823505 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.823514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.824161 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.824165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.824282 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.824759 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:36.824774 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:36.826223 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:36.828857 (+ 2634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.828860 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.828862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.829059 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.829883 (+ 824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.829886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.829888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.829982 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.830436 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:36.830442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:36.830565 (+ 123us) client_negotiation.cc:769] Sending connection context
0620 14:57:36.830663 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 14:57:36.830797 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260620 14:57:36.831676 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.820834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36228 (local address 127.7.24.252:43901)
0620 14:57:36.821146 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.821157 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.821173 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:57:36.821210 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.821212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.821252 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.821437 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.822151 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.822820 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.823657 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.824019 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.824151 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.826359 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.826363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.826365 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.826389 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.828723 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.829671 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.829677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.829682 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.829768 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.830104 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.830107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.830109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.830326 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.831269 (+ 943us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.831352 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.831517 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":94,"threads_started":1}
I20260620 14:57:36.834571 29615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369: Bootstrap starting.
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:36.836331 29690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } attempt: 3
I20260620 14:57:36.837471 29615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:36.838311 29615 log.cc:824] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:36.840724 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.830046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.7.24.254:46587)
0620 14:57:36.830154 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.830158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.830307 (+ 149us) server_negotiation.cc:407] Connection header received
0620 14:57:36.830497 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.830500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.830543 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.830618 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.831469 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.832107 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.833054 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.833279 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.833401 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.836171 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.836175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.836213 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.836241 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.838582 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.839127 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.839133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.839139 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.839250 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.839595 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.839600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.839602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.839823 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.839979 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.840412 (+ 433us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.840540 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
I20260620 14:57:36.842020 29615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369: No bootstrap required, opened a new log
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
I20260620 14:57:36.846143 29615 raft_consensus.cc:348] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } }
I20260620 14:57:36.846395 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.829830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:45493 (local address 127.0.0.1:35576)
0620 14:57:36.830023 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.830038 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:36.830114 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:36.830341 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:36.830344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:36.830543 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:36.830662 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.830669 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.831515 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.831521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.832264 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.832276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.839314 (+ 7038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.839318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.839428 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.839878 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:36.839893 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:36.841900 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:36.844268 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.844273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.844275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.844636 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.845038 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.845042 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.845044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.845153 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.845802 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:36.845806 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:36.845920 (+ 114us) client_negotiation.cc:769] Sending connection context
0620 14:57:36.846056 (+ 136us) client_negotiation.cc:240] Negotiation successful
0620 14:57:36.846211 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260620 14:57:36.846400 29615 raft_consensus.cc:729] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b005e650e15d434cad5a0e3b89278369, State: Initialized, Role: FOLLOWER
I20260620 14:57:36.846525 29692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.829910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35576 (local address 127.7.24.253:45493)
0620 14:57:36.830018 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.830023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.830079 (+ 56us) server_negotiation.cc:407] Connection header received
0620 14:57:36.830188 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.830191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.830234 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.830331 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.830764 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.831407 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.838796 (+ 7389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.839186 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.839308 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.842051 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.842056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.842059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.842091 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.844160 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.844785 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.844792 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.844797 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.844916 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.845286 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.845291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.845294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.845624 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.845749 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.846191 (+ 442us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.846311 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":31}
I20260620 14:57:36.847186 29765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:57:36.847141 29615 consensus_queue.cc:260] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } }
I20260620 14:57:36.848307 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.836738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36234)
0620 14:57:36.836937 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.836960 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:36.837054 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:36.837456 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:36.837462 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:36.837704 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:36.837844 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.837854 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.839171 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.839177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.840070 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.840081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.840787 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.840792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.840892 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.841648 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:36.841664 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:36.843713 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:36.845918 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.845921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.845923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.846216 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.847165 (+ 949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.847168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.847170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.847269 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.847804 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:36.847807 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:36.847903 (+ 96us) client_negotiation.cc:769] Sending connection context
0620 14:57:36.847996 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 14:57:36.848142 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"spinlock_wait_cycles":53504}
I20260620 14:57:36.848796 29615 sys_catalog.cc:564] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:36.848932 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.837043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.7.24.252:43901)
0620 14:57:36.837180 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.837185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.837199 (+ 14us) server_negotiation.cc:407] Connection header received
0620 14:57:36.837241 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.837245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.837301 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.837987 (+ 686us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.838107 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.839032 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.840222 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.840622 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.841122 (+ 500us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.843870 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.843875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.843878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.843910 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.845802 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.846335 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.846338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.846340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.846964 (+ 624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.847396 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.847400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.847402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.847654 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.848577 (+ 923us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.848654 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.848784 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 14:57:36.851291 29773 sys_catalog.cc:455] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } } }
I20260620 14:57:36.851869 29773 sys_catalog.cc:458] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:36.852322 29690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc: Bootstrap starting.
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
Jun 20 14:57:36 dist-test-slave-sms9 krb5kdc[29534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
I20260620 14:57:36.855468 29690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:36.856467 29690 log.cc:824] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:36.857949 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.847629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:43901 (local address 127.0.0.1:36248)
0620 14:57:36.847941 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:36.847955 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:36.848028 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:36.848299 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:36.848302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:36.848700 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:36.848788 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.848795 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.850039 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.850042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.850884 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:36.850894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.851560 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.851564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:36.851632 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.852275 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:36.852289 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:36.854394 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:36.856367 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.856370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.856384 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.856714 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.857003 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:36.857005 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:36.857007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:36.857076 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:36.857472 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:36.857476 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:36.857553 (+ 77us) client_negotiation.cc:769] Sending connection context
0620 14:57:36.857649 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 14:57:36.857786 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249}
I20260620 14:57:36.854208 29547 master_runner.cc:186] Error getting master registration for 127.7.24.253:45493: OK, instance_id { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" instance_seqno: 1781967456582797 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 14:57:36.858707 29547 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 14:57:36.858707 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.847802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.7.24.252:43901)
0620 14:57:36.847911 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.847916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.848098 (+ 182us) server_negotiation.cc:407] Connection header received
0620 14:57:36.848123 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.848125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.848156 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.848216 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.848947 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.849911 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.851007 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.851343 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.851457 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.854551 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.854555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.854559 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.854581 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.856249 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.856827 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.856831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.856833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.856915 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.857155 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.857158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.857159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.857350 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.857441 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.858338 (+ 897us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.858503 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 14:57:36.859606 29690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc: No bootstrap required, opened a new log
I20260620 14:57:36.860225 29692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.843199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35582 (local address 127.7.24.253:45493)
0620 14:57:36.846869 (+ 3670us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.846872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.846882 (+ 10us) server_negotiation.cc:407] Connection header received
0620 14:57:36.846910 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.846912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.846987 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.847069 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.848173 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.848954 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.850154 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.850538 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.850616 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.855993 (+ 5377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.855997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.856000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.856027 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.858043 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.858647 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.858651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.858653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.858746 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.859122 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.859125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.859126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.859384 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.859556 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.859972 (+ 416us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.860090 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3620,"thread_start_us":108,"threads_started":1}
W20260620 14:57:36.860809 29784 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:57:36.860901 29784 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:57:36.861740 29765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40: Bootstrap starting.
I20260620 14:57:36.864113 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:36.864319 29765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:36.865181 29765 log.cc:824] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:36.865561 29589 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b005e650e15d434cad5a0e3b89278369"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48766
I20260620 14:57:36.865844 29589 raft_consensus.cc:482] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 14:57:36.865715 29690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } }
I20260620 14:57:36.865969 29589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:36.865986 29690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2852623339b8488e9365dfb1a4c36abc, State: Initialized, Role: FOLLOWER
I20260620 14:57:36.866290 29690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } }
I20260620 14:57:36.867501 29690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:36.867447 29589 raft_consensus.cc:504] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } }
I20260620 14:57:36.869168 29664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b005e650e15d434cad5a0e3b89278369" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2852623339b8488e9365dfb1a4c36abc"
I20260620 14:57:36.869401 29664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:36.870811 29664 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b005e650e15d434cad5a0e3b89278369 in term 1.
I20260620 14:57:36.871094 29589 leader_election.cc:288] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [CANDIDATE]: Term 1 election: Requested vote from peers 2852623339b8488e9365dfb1a4c36abc (127.7.24.253:45493), 747905212037498996bd535b1190aa40 (127.7.24.252:43901)
I20260620 14:57:36.871507 29563 leader_election.cc:302] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2852623339b8488e9365dfb1a4c36abc, b005e650e15d434cad5a0e3b89278369; no voters:
I20260620 14:57:36.871841 29773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:36.872076 29773 raft_consensus.cc:686] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [term 1 LEADER]: Becoming Leader. State: Replica: b005e650e15d434cad5a0e3b89278369, State: Running, Role: LEADER
I20260620 14:57:36.872439 29773 consensus_queue.cc:237] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } }
W20260620 14:57:36.874919 29799 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:57:36.875008 29799 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:57:36.876852 29773 sys_catalog.cc:455] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b005e650e15d434cad5a0e3b89278369. Latest consensus state: current_term: 1 leader_uuid: "b005e650e15d434cad5a0e3b89278369" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } } }
I20260620 14:57:36.876986 29773 sys_catalog.cc:458] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:36.883155 29807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:36.884254 29807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:36.882498 29622 master_runner.cc:186] Error getting master registration for 127.7.24.252:43901: OK, instance_id { permanent_uuid: "747905212037498996bd535b1190aa40" instance_seqno: 1781967456804201 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 14:57:36.886059 29622 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 14:57:36.887219 29788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } } }
I20260620 14:57:36.887393 29788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:36.889326 29765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40: No bootstrap required, opened a new log
I20260620 14:57:36.892414 29664 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [term 1 FOLLOWER]: Refusing update from remote peer b005e650e15d434cad5a0e3b89278369: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:36.893183 29773 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:36.893986 29765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } }
I20260620 14:57:36.894280 29765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 747905212037498996bd535b1190aa40, State: Initialized, Role: FOLLOWER
I20260620 14:57:36.894670 29765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } }
I20260620 14:57:36.896154 29765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:36.897372 29739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b005e650e15d434cad5a0e3b89278369" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "747905212037498996bd535b1190aa40"
I20260620 14:57:36.897611 29739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:36.899004 29739 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b005e650e15d434cad5a0e3b89278369 in term 1.
I20260620 14:57:36.899524 29738 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [term 1 FOLLOWER]: Refusing update from remote peer b005e650e15d434cad5a0e3b89278369: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:36.900703 29773 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [LEADER]: Connected to new peer: Peer: permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:36.901074 29788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b005e650e15d434cad5a0e3b89278369. Latest consensus state: current_term: 1 leader_uuid: "b005e650e15d434cad5a0e3b89278369" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } } }
I20260620 14:57:36.901201 29788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:36.901795 29812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } } }
I20260620 14:57:36.901921 29812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:36.906042 29812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b005e650e15d434cad5a0e3b89278369. Latest consensus state: current_term: 1 leader_uuid: "b005e650e15d434cad5a0e3b89278369" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } } }
I20260620 14:57:36.906178 29812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:36.906484 29800 sys_catalog.cc:455] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b005e650e15d434cad5a0e3b89278369" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } } }
I20260620 14:57:36.906633 29800 sys_catalog.cc:458] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:36.907132 29788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b005e650e15d434cad5a0e3b89278369" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } } }
I20260620 14:57:36.907265 29788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:36.907437 29811 mvcc.cc:205] Tried to move back new op lower bound from 7298938703418847232 to 7298938703353769984. Current Snapshot: MvccSnapshot[applied={T|T < 7298938703418847232}]
I20260620 14:57:36.907598 29813 mvcc.cc:205] Tried to move back new op lower bound from 7298938703418847232 to 7298938703353769984. Current Snapshot: MvccSnapshot[applied={T|T < 7298938703418847232}]
W20260620 14:57:36.909356 29825 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:57:36.909448 29825 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:57:36.913537 29773 sys_catalog.cc:455] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b005e650e15d434cad5a0e3b89278369" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } } }
I20260620 14:57:36.913705 29773 sys_catalog.cc:458] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:36.914206 29807 catalog_manager.cc:1347] Generated new cluster ID: d6bb990295bb41aaa901f650ea7d49cf
I20260620 14:57:36.914280 29807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:36.914502 29692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.877103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.7.24.253:45493)
0620 14:57:36.877220 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.877225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.883535 (+ 6310us) server_negotiation.cc:407] Connection header received
0620 14:57:36.886720 (+ 3185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.886725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.886790 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.886856 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.888077 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.888864 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.898596 (+ 9732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.898982 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.899109 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.907651 (+ 8542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.907656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.907659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.907693 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.909627 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.910217 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.910224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.910229 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.910324 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.912127 (+ 1803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.912131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.912133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.912393 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.912520 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.913831 (+ 1311us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.913987 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 14:57:36.915405 29812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b005e650e15d434cad5a0e3b89278369" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b005e650e15d434cad5a0e3b89278369" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 46587 } } peers { permanent_uuid: "2852623339b8488e9365dfb1a4c36abc" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 45493 } } peers { permanent_uuid: "747905212037498996bd535b1190aa40" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 43901 } } }
I20260620 14:57:36.915544 29812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:36.915874 29826 mvcc.cc:205] Tried to move back new op lower bound from 7298938703418847232 to 7298938703353769984. Current Snapshot: MvccSnapshot[applied={T|T < 7298938703418847232}]
I20260620 14:57:36.917797 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.878178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36264 (local address 127.7.24.252:43901)
0620 14:57:36.878298 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.878303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.889214 (+ 10911us) server_negotiation.cc:407] Connection header received
0620 14:57:36.890576 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.890579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.890629 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.890682 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.900082 (+ 9400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.900850 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.903567 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.903920 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.904005 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.913182 (+ 9177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.913186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.913189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.913216 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.915042 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.916632 (+ 1590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.916636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.916637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.916717 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.916971 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.916974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.916975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.917158 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.917269 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.917554 (+ 285us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.917663 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:57:36.924257 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:36.882820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.7.24.254:46587)
0620 14:57:36.882945 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.882951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:36.913266 (+ 30315us) server_negotiation.cc:407] Connection header received
0620 14:57:36.914996 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:36.915000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:36.915059 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:36.915116 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:36.916077 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.916940 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.918220 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:36.918619 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:36.918711 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:36.920571 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:36.920575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:36.920577 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:36.920603 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:36.922234 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.922800 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.922804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.922806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.922899 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:36.923215 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:36.923219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:36.923221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:36.923450 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:36.923558 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:36.923963 (+ 405us) server_negotiation.cc:299] Negotiation successful
0620 14:57:36.924090 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:57:36.939790 29807 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:36.943603 29807 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:36.964879 29807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b005e650e15d434cad5a0e3b89278369: Generated new TSK 0
I20260620 14:57:36.965890 29807 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 14:57:37.002198 29768 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:36.999438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36268 (local address 127.7.24.252:43901)
0620 14:57:36.999578 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:36.999583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.001000 (+ 1417us) server_negotiation.cc:407] Connection header received
0620 14:57:37.001239 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.001244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.001297 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.001357 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.002034 (+ 677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36268: BlockingRecv error: recv got EOF from 127.0.0.1:36268 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260620 14:57:37.003723 29619 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.000369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48780 (local address 127.7.24.254:46587)
0620 14:57:37.000490 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.000495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.002730 (+ 2235us) server_negotiation.cc:407] Connection header received
0620 14:57:37.002914 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.002918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.002970 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.003025 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.003583 (+ 558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48780: BlockingRecv error: recv got EOF from 127.0.0.1:48780 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260620 14:57:37.005810 29692 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.004249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.7.24.253:45493)
0620 14:57:37.004371 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.004376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.004858 (+ 482us) server_negotiation.cc:407] Connection header received
0620 14:57:37.005035 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.005038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.005089 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.005143 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.005668 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35604: BlockingRecv error: recv got EOF from 127.0.0.1:35604 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260620 14:57:37.010587 29692 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.008960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.7.24.253:45493)
0620 14:57:37.009070 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.009074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.009512 (+ 438us) server_negotiation.cc:407] Connection header received
0620 14:57:37.009744 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.009748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.009829 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.009886 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.010438 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35616: BlockingRecv error: recv got EOF from 127.0.0.1:35616 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260620 14:57:37.015967 29692 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.014352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35632 (local address 127.7.24.253:45493)
0620 14:57:37.014497 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.014502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.015000 (+ 498us) server_negotiation.cc:407] Connection header received
0620 14:57:37.015219 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.015223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.015278 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.015334 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.015856 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35632: BlockingRecv error: recv got EOF from 127.0.0.1:35632 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 14:57:37.021864 29692 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.020309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.7.24.253:45493)
0620 14:57:37.020421 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.020426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.020910 (+ 484us) server_negotiation.cc:407] Connection header received
0620 14:57:37.021108 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.021112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.021165 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.021236 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.021754 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35640: BlockingRecv error: recv got EOF from 127.0.0.1:35640 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260620 14:57:37.029922 29692 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.028468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.7.24.253:45493)
0620 14:57:37.028599 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.028603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.029051 (+ 448us) server_negotiation.cc:407] Connection header received
0620 14:57:37.029246 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.029252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.029300 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.029350 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.029820 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35646: BlockingRecv error: recv got EOF from 127.0.0.1:35646 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260620 14:57:37.042145 29692 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.040352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35656 (local address 127.7.24.253:45493)
0620 14:57:37.040470 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.040475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.041012 (+ 537us) server_negotiation.cc:407] Connection header received
0620 14:57:37.041363 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.041367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.041423 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.041477 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.042035 (+ 558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35656: BlockingRecv error: recv got EOF from 127.0.0.1:35656 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260620 14:57:37.062726 29692 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.060799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35672 (local address 127.7.24.253:45493)
0620 14:57:37.060940 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.060945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.061538 (+ 593us) server_negotiation.cc:407] Connection header received
0620 14:57:37.061778 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.061791 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.061856 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.061914 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.062603 (+ 689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35672: BlockingRecv error: recv got EOF from 127.0.0.1:35672 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260620 14:57:37.099159 29692 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.097758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35688 (local address 127.7.24.253:45493)
0620 14:57:37.097899 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.097903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.098222 (+ 319us) server_negotiation.cc:407] Connection header received
0620 14:57:37.098390 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.098394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.098453 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.098573 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.099038 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35688: BlockingRecv error: recv got EOF from 127.0.0.1:35688 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260620 14:57:37.166878 29692 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.165622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35698 (local address 127.7.24.253:45493)
0620 14:57:37.165791 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.165794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.165882 (+ 88us) server_negotiation.cc:407] Connection header received
0620 14:57:37.166053 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.166056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.166111 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.166202 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.166734 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35698: BlockingRecv error: recv got EOF from 127.0.0.1:35698 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260620 14:57:37.298532 29692 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.297219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35712 (local address 127.7.24.253:45493)
0620 14:57:37.297409 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.297416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.297669 (+ 253us) server_negotiation.cc:407] Connection header received
0620 14:57:37.297805 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.297808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.297861 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.297937 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.298374 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35712: BlockingRecv error: recv got EOF from 127.0.0.1:35712 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260620 14:57:37.557946 29692 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:37.556829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35716 (local address 127.7.24.253:45493)
0620 14:57:37.556980 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:37.556987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:37.557011 (+ 24us) server_negotiation.cc:407] Connection header received
0620 14:57:37.557198 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:37.557202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:37.557272 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:37.557367 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:37.557824 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35716: BlockingRecv error: recv got EOF from 127.0.0.1:35716 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 14:57:37.876324 29799 catalog_manager.cc:1259] Loaded cluster ID: d6bb990295bb41aaa901f650ea7d49cf
I20260620 14:57:37.876446 29799 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc: loading cluster ID for follower catalog manager: success
I20260620 14:57:37.878059 29799 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc: acquiring CA information for follower catalog manager: success
I20260620 14:57:37.879199 29799 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2852623339b8488e9365dfb1a4c36abc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 14:57:37.910884 29825 catalog_manager.cc:1259] Loaded cluster ID: d6bb990295bb41aaa901f650ea7d49cf
I20260620 14:57:37.910992 29825 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40: loading cluster ID for follower catalog manager: success
I20260620 14:57:37.912645 29825 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40: acquiring CA information for follower catalog manager: success
I20260620 14:57:37.913763 29825 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 747905212037498996bd535b1190aa40: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 14:57:38.076713 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:38.072420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.7.24.253:45493)
0620 14:57:38.072710 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:38.072715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:38.072835 (+ 120us) server_negotiation.cc:407] Connection header received
0620 14:57:38.073014 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:38.073018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:38.073155 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:38.073337 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:38.073613 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:38.074157 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:38.075131 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:38.075385 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:38.075501 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:38.075858 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:38.076306 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:38.076447 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:38.076488 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 14:57:38.076545 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":69,"threads_started":1}
I20260620 14:57:38.083846 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:38.079965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36278 (local address 127.7.24.252:43901)
0620 14:57:38.080188 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:38.080191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:38.080204 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:57:38.080331 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:38.080334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:38.080475 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:38.080576 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:38.080785 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:38.081388 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:38.082277 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:38.082583 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:38.082698 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:38.083020 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:38.083448 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:38.083588 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:38.083626 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 14:57:38.083683 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":88,"threads_started":1}
I20260620 14:57:38.091179 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:38.086997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35724 (local address 127.7.24.253:45493)
0620 14:57:38.087219 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:38.087227 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:38.087571 (+ 344us) server_negotiation.cc:407] Connection header received
0620 14:57:38.087671 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:38.087674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:38.087715 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:38.087814 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:38.088080 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:38.088876 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:38.089915 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:38.090196 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:38.090283 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:38.090644 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:38.090724 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:38.090785 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:38.090976 (+ 191us) server_negotiation.cc:299] Negotiation successful
0620 14:57:38.091030 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260620 14:57:38.091351 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:38.087073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.7.24.252:43901)
0620 14:57:38.087191 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:38.087198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:38.087855 (+ 657us) server_negotiation.cc:407] Connection header received
0620 14:57:38.088035 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:38.088039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:38.088095 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:38.088167 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:38.088408 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:38.089143 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:38.090251 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:38.090504 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:38.090621 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:38.090882 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:38.090968 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:38.091034 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:38.091174 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 14:57:38.091233 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 14:57:38.091955 29910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:38.087323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48786 (local address 127.7.24.254:46587)
0620 14:57:38.087581 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:38.087585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:38.087600 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:57:38.087700 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:38.087703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:38.087879 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:38.088019 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:38.088274 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:38.088853 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:38.090081 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:38.090425 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:38.090565 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:38.090996 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:38.091520 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:38.091611 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:38.091737 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 14:57:38.091801 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
I20260620 14:57:38.095381 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 29547
I20260620 14:57:38.101732 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 29622
I20260620 14:57:38.107689 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 29697
2026-06-20T14:57:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3109 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 14:57:38 dist-test-slave-sms9 krb5kdc[29913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 14:57:38 dist-test-slave-sms9 krb5kdc[29913](info): set up 2 sockets
Jun 20 14:57:38 dist-test-slave-sms9 krb5kdc[29913](info): commencing operation
krb5kdc: starting...
W20260620 14:57:39.162999 7267 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/test-user.keytab.
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T14:57:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T14:57:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.24.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
Entry for principal kudu/127.7.24.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab.
I20260620 14:57:39.294214 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:38241
--webserver_interface=127.7.24.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:45721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:38241,127.7.24.253:36451,127.7.24.252:46323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 14:57:39.413703 29926 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 14:57:39.414037 29926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:39.414064 29926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:39.414176 29926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 14:57:39.418820 29926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:39.418898 29926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:39.418920 29926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:39.418939 29926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:39.418957 29926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:39.425019 29926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:45721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:38241,127.7.24.253:36451,127.7.24.252:46323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.254.keytab
--principal=kudu/127.7.24.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.254:38241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.24.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:39.426290 29926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:39.427424 29926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 14:57:39.436111 29926 server_base.cc:1034] running on GCE node
W20260620 14:57:39.436115 29931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:39.436100 29934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:39.436115 29932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:39.436865 29926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:39.437986 29926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:39.439211 29926 hybrid_clock.cc:648] HybridClock initialized: now 1781967459439173 us; error 43 us; skew 500 ppm
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:39.442152 29926 init.cc:377] Logged in from keytab as kudu/127.7.24.254@KRBTEST.COM (short username kudu)
I20260620 14:57:39.443121 29926 webserver.cc:460] Webserver started at http://127.7.24.254:39735/ using document root <none> and password file <none>
I20260620 14:57:39.443689 29926 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:39.443809 29926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:39.444041 29926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:39.446094 29926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/data/instance:
uuid: "af5b6769810a4c7ab902fe2d40ad8629"
format_stamp: "Formatted at 2026-06-20 14:57:39 on dist-test-slave-sms9"
server_key: "077e520a12e4d8befae5b8256b396fae"
server_key_iv: "5627dda07a488da0043680dbd8f2a532"
server_key_version: "encryptionkey@0"
I20260620 14:57:39.446707 29926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/wal/instance:
uuid: "af5b6769810a4c7ab902fe2d40ad8629"
format_stamp: "Formatted at 2026-06-20 14:57:39 on dist-test-slave-sms9"
server_key: "077e520a12e4d8befae5b8256b396fae"
server_key_iv: "5627dda07a488da0043680dbd8f2a532"
server_key_version: "encryptionkey@0"
I20260620 14:57:39.450371 29926 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 14:57:39.452742 29941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:39.453959 29926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260620 14:57:39.454108 29926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/wal
uuid: "af5b6769810a4c7ab902fe2d40ad8629"
format_stamp: "Formatted at 2026-06-20 14:57:39 on dist-test-slave-sms9"
server_key: "077e520a12e4d8befae5b8256b396fae"
server_key_iv: "5627dda07a488da0043680dbd8f2a532"
server_key_version: "encryptionkey@0"
I20260620 14:57:39.454227 29926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:39.473762 29926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:39.477172 29926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:39.477386 29926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:39.484265 29926 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.254:38241
I20260620 14:57:39.484545 29993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.254:38241 every 8 connection(s)
I20260620 14:57:39.485212 29926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/data/info.pb
I20260620 14:57:39.487905 29994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:39.491358 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 29926
I20260620 14:57:39.491469 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-0/wal/instance
I20260620 14:57:39.491770 7267 external_mini_cluster.cc:1428] Setting key 2d54782038cef294d0cf920f41134584
I20260620 14:57:39.491603 29994 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.24.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.253@KRBTEST.COM" created.
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:39.510645 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.494028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48270 (local address 127.7.24.254:38241)
0620 14:57:39.494581 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.494599 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.494805 (+ 206us) server_negotiation.cc:407] Connection header received
0620 14:57:39.495985 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.495989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.496177 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.496383 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.497443 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.499036 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.500713 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.501398 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.501573 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.505374 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.505381 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.505396 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.505423 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.507311 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.507909 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.507915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.507916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.508029 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.508415 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.508418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.508419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.508895 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.509029 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.509500 (+ 471us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.509772 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":351,"thread_start_us":165,"threads_started":1}
I20260620 14:57:39.510648 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.494028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38241 (local address 127.0.0.1:48270)
0620 14:57:39.494657 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.494695 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:39.495599 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:39.496381 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:39.496386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:39.496846 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:39.497265 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.497304 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.499186 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.499192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.500533 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.500545 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.501831 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.501837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.501987 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.502748 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:39.502770 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:39.505187 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:39.507434 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.507446 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.507459 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.507773 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.508137 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.508143 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.508148 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.508312 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.509039 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:39.509049 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:39.509226 (+ 177us) client_negotiation.cc:769] Sending connection context
0620 14:57:39.509440 (+ 214us) client_negotiation.cc:240] Negotiation successful
0620 14:57:39.509886 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":101,"spinlock_wait_cycles":51968,"thread_start_us":164,"threads_started":1}
I20260620 14:57:39.513401 29994 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:57:39.514137 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.513894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:36451 (local address 127.0.0.1:40572)
0620 14:57:39.514010 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.514079 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:36451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260620 14:57:39.514412 29944 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.24.253:36451: connect: Connection refused (error 111)
I20260620 14:57:39.516288 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.516022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:36451 (local address 127.0.0.1:40578)
0620 14:57:39.516137 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.516225 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:36451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260620 14:57:39.516464 29994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.253:36451: Network error: Client connection negotiation failed: client connection to 127.7.24.253:36451: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab.
Entry for principal kudu/127.7.24.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab.
I20260620 14:57:39.522569 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.253:36451
--webserver_interface=127.7.24.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab
--principal=kudu/127.7.24.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:45721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:38241,127.7.24.253:36451,127.7.24.252:46323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:57:39.565721 29994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } attempt: 1
I20260620 14:57:39.566762 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.566367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:36451 (local address 127.0.0.1:40592)
0620 14:57:39.566614 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.566679 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:36451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260620 14:57:39.567566 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.567330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:36451 (local address 127.0.0.1:40604)
0620 14:57:39.567445 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.567504 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:36451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 14:57:39.567703 29994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.253:36451: Network error: Client connection negotiation failed: client connection to 127.7.24.253:36451: connect: Connection refused (error 111)
W20260620 14:57:39.632292 30001 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 14:57:39.632622 30001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:39.632701 30001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:39.632783 30001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260620 14:57:39.635871 29994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } attempt: 2
I20260620 14:57:39.636780 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.636470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:36451 (local address 127.0.0.1:40620)
0620 14:57:39.636613 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.636681 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:36451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260620 14:57:39.636930 30001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:39.637012 30001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:39.637081 30001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:39.637121 30001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:39.637158 30001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:39.637519 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.637303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:36451 (local address 127.0.0.1:40624)
0620 14:57:39.637412 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.637461 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.253:36451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260620 14:57:39.637655 29994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.253:36451: Network error: Client connection negotiation failed: client connection to 127.7.24.253:36451: connect: Connection refused (error 111)
I20260620 14:57:39.642606 30001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:45721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:38241,127.7.24.253:36451,127.7.24.252:46323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.253.keytab
--principal=kudu/127.7.24.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.253:36451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.24.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:39.643754 30001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:39.644770 30001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:39.653365 30007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:39.653401 30001 server_base.cc:1034] running on GCE node
W20260620 14:57:39.653365 30006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:39.653379 30009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:39.654084 30001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:39.655225 30001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:39.656412 30001 hybrid_clock.cc:648] HybridClock initialized: now 1781967459656378 us; error 45 us; skew 500 ppm
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:39.659622 30001 init.cc:377] Logged in from keytab as kudu/127.7.24.253@KRBTEST.COM (short username kudu)
I20260620 14:57:39.660516 30001 webserver.cc:460] Webserver started at http://127.7.24.253:38015/ using document root <none> and password file <none>
I20260620 14:57:39.661103 30001 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:39.661196 30001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:39.661437 30001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:39.663532 30001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/data/instance:
uuid: "8aabf7cd1dc1442e8f0247ff9d451f44"
format_stamp: "Formatted at 2026-06-20 14:57:39 on dist-test-slave-sms9"
server_key: "a8bf658e09ddaa87ed76e151987f5e8e"
server_key_iv: "a3c3a7ecc3a4b136c5e895c59836ceb7"
server_key_version: "encryptionkey@0"
I20260620 14:57:39.664144 30001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/wal/instance:
uuid: "8aabf7cd1dc1442e8f0247ff9d451f44"
format_stamp: "Formatted at 2026-06-20 14:57:39 on dist-test-slave-sms9"
server_key: "a8bf658e09ddaa87ed76e151987f5e8e"
server_key_iv: "a3c3a7ecc3a4b136c5e895c59836ceb7"
server_key_version: "encryptionkey@0"
I20260620 14:57:39.667968 30001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 14:57:39.670408 30016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:39.672003 30001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 14:57:39.672166 30001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/wal
uuid: "8aabf7cd1dc1442e8f0247ff9d451f44"
format_stamp: "Formatted at 2026-06-20 14:57:39 on dist-test-slave-sms9"
server_key: "a8bf658e09ddaa87ed76e151987f5e8e"
server_key_iv: "a3c3a7ecc3a4b136c5e895c59836ceb7"
server_key_version: "encryptionkey@0"
I20260620 14:57:39.672291 30001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:39.693030 30001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:39.696228 30001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:39.696517 30001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:39.704006 30001 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.253:36451
I20260620 14:57:39.704365 30068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.253:36451 every 8 connection(s)
I20260620 14:57:39.705019 30001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/data/info.pb
I20260620 14:57:39.707880 30069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:39.709040 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 30001
I20260620 14:57:39.709193 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-1/wal/instance
I20260620 14:57:39.709550 7267 external_mini_cluster.cc:1428] Setting key 82954fa423f780adc75ccb7bb25574a4
I20260620 14:57:39.711861 30069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.24.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.24.252@KRBTEST.COM" created.
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:39.726565 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.713947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48282 (local address 127.7.24.254:38241)
0620 14:57:39.714093 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.714097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.715196 (+ 1099us) server_negotiation.cc:407] Connection header received
0620 14:57:39.716145 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.716149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.716209 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.716328 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.717505 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.718041 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.719345 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.719606 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.719750 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.722285 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.722289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.722291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.722324 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.724174 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.724777 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.724783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.724788 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.724890 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.725280 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.725286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.725290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.725537 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.725649 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.726188 (+ 539us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.726342 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 14:57:39.727355 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.714423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38241 (local address 127.0.0.1:48282)
0620 14:57:39.715023 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.715065 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:39.715941 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:39.716490 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:39.716497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:39.716845 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:39.717360 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.717375 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.718288 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.718291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.719220 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.719230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.719798 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.719803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.720040 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.720738 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:39.720755 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:39.722118 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:39.724373 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.724378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.724390 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.724655 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.725016 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.725018 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.725020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.725166 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.725654 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:39.725659 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:39.725901 (+ 242us) client_negotiation.cc:769] Sending connection context
0620 14:57:39.726075 (+ 174us) client_negotiation.cc:240] Negotiation successful
0620 14:57:39.726386 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":160,"threads_started":1}
I20260620 14:57:39.728824 29994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } attempt: 3
I20260620 14:57:39.728988 30069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.24.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab.
Entry for principal kudu/127.7.24.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab.
I20260620 14:57:39.737773 7267 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
/tmp/dist-test-taskNGdDYa/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.252:46323
--webserver_interface=127.7.24.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab
--principal=kudu/127.7.24.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.24.212:45721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.24.254:38241,127.7.24.253:36451,127.7.24.252:46323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 14:57:39.740202 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.729496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:36451 (local address 127.0.0.1:40644)
0620 14:57:39.729651 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.729667 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:39.729744 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:39.730842 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:39.730846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:39.731017 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:39.731114 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.731120 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.732205 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.732208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.732894 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.732903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.733724 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.733728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.733875 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.734430 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:39.734443 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:39.736136 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:39.738355 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.738361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.738365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.738633 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.738921 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.738925 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.738927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.739010 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.739640 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:39.739645 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:39.739745 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 14:57:39.739846 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 14:57:39.739991 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":44,"spinlock_wait_cycles":768}
I20260620 14:57:39.740476 30075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.729567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40636 (local address 127.7.24.253:36451)
0620 14:57:39.730004 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.730012 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.730030 (+ 18us) server_negotiation.cc:407] Connection header received
0620 14:57:39.730062 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.730066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.730202 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.730347 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.730883 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.731904 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.733318 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.733585 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.733696 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.735739 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.735747 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.735758 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.735800 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.738515 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.739065 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.739071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.739076 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.739186 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.739538 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.739541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.739542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.739720 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.739836 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.740143 (+ 307us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.740286 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":363,"spinlock_wait_cycles":1792,"thread_start_us":161,"threads_started":1}
I20260620 14:57:39.740751 30074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.729579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40644 (local address 127.7.24.253:36451)
0620 14:57:39.730603 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.730609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.730618 (+ 9us) server_negotiation.cc:407] Connection header received
0620 14:57:39.730642 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.730644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.730671 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.730817 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.731234 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.732077 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.733002 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.733595 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.733714 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.736271 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.736275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.736277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.736306 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.738229 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.738738 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.738744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.738746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.738829 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.739097 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.739102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.739104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.739467 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.739600 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.740509 (+ 909us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.740626 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":971,"spinlock_wait_cycles":17152,"thread_start_us":121,"threads_started":1}
I20260620 14:57:39.741029 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.729192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:36451 (local address 127.0.0.1:40636)
0620 14:57:39.729336 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.729353 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:39.729439 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:39.730406 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:39.730412 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:39.730617 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:39.730708 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.730733 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.732022 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.732025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.733169 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.733179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.733701 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.733704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.733810 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.734251 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:39.734262 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:39.735590 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:39.738643 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.738649 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.738654 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.738933 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.739306 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.739311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.739315 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.739420 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.739839 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:39.739843 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:39.739943 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 14:57:39.740052 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 14:57:39.740186 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260620 14:57:39.742609 30069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:57:39.742683 29994 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } has no permanent_uuid. Determining permanent_uuid...
I20260620 14:57:39.743371 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.743137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57476)
0620 14:57:39.743245 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.743321 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 14:57:39.743542 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.743192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57488)
0620 14:57:39.743398 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.743458 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260620 14:57:39.743571 30020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
I20260620 14:57:39.744298 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.744005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57490)
0620 14:57:39.744188 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.744236 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260620 14:57:39.744462 29994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:46323: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
I20260620 14:57:39.744510 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.744242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57494)
0620 14:57:39.744399 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.744444 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260620 14:57:39.744638 30069 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:46323: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
I20260620 14:57:39.775635 29994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } attempt: 1
I20260620 14:57:39.776610 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.776278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57498)
0620 14:57:39.776462 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.776528 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260620 14:57:39.777392 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.777151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57508)
0620 14:57:39.777280 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.777334 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260620 14:57:39.777513 29994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:46323: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
I20260620 14:57:39.793891 30069 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } attempt: 1
I20260620 14:57:39.794831 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.794530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57510)
0620 14:57:39.794682 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.794749 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260620 14:57:39.795642 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.795398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57516)
0620 14:57:39.795529 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.795587 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260620 14:57:39.795797 30069 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:46323: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
W20260620 14:57:39.852483 30076 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
I20260620 14:57:39.852701 29994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } attempt: 2
W20260620 14:57:39.852876 30076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 14:57:39.852945 30076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 14:57:39.853022 30076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260620 14:57:39.853578 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.853276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57526)
0620 14:57:39.853437 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.853500 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 14:57:39.854368 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.854142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57542)
0620 14:57:39.854266 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.854314 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260620 14:57:39.854578 29994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:46323: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
W20260620 14:57:39.857681 30076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 14:57:39.857755 30076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 14:57:39.857796 30076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 14:57:39.857832 30076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 14:57:39.857861 30076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 14:57:39.863451 30076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.24.212:45721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.24.254:38241,127.7.24.253:36451,127.7.24.252:46323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/krb5kdc/kudu_127.7.24.252.keytab
--principal=kudu/127.7.24.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.24.252:46323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.24.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 20 Jun 2026 14:09:29 UTC on ccycloud.quasar-vozkmi.root.comops.site
build id 4457508
I20260620 14:57:39.863955 30069 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } attempt: 2
I20260620 14:57:39.864645 30076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 14:57:39.864771 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.864471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57558)
0620 14:57:39.864602 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.864664 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260620 14:57:39.865507 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.865289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57574)
0620 14:57:39.865399 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.865445 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260620 14:57:39.865675 30069 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.24.252:46323: Network error: Client connection negotiation failed: client connection to 127.7.24.252:46323: connect: Connection refused (error 111)
I20260620 14:57:39.865888 30076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 14:57:39.874321 30085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:39.874318 30082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 14:57:39.874326 30083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 14:57:39.874768 30076 server_base.cc:1034] running on GCE node
I20260620 14:57:39.875492 30076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 14:57:39.876705 30076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 14:57:39.877908 30076 hybrid_clock.cc:648] HybridClock initialized: now 1781967459877888 us; error 56 us; skew 500 ppm
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 14:57:39.880887 30076 init.cc:377] Logged in from keytab as kudu/127.7.24.252@KRBTEST.COM (short username kudu)
I20260620 14:57:39.881815 30076 webserver.cc:460] Webserver started at http://127.7.24.252:37157/ using document root <none> and password file <none>
I20260620 14:57:39.882416 30076 fs_manager.cc:359] Metadata directory not provided
I20260620 14:57:39.882527 30076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 14:57:39.882768 30076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 14:57:39.884802 30076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/data/instance:
uuid: "a43617f1c5b644cdb07cc10969e5dae7"
format_stamp: "Formatted at 2026-06-20 14:57:39 on dist-test-slave-sms9"
server_key: "e5122f9d2eb3f9a79d7e0b8a23197bae"
server_key_iv: "5ad925ef9b972d085845581e9c8edfe1"
server_key_version: "encryptionkey@0"
I20260620 14:57:39.885362 30076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/wal/instance:
uuid: "a43617f1c5b644cdb07cc10969e5dae7"
format_stamp: "Formatted at 2026-06-20 14:57:39 on dist-test-slave-sms9"
server_key: "e5122f9d2eb3f9a79d7e0b8a23197bae"
server_key_iv: "5ad925ef9b972d085845581e9c8edfe1"
server_key_version: "encryptionkey@0"
I20260620 14:57:39.889094 30076 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260620 14:57:39.891441 30092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 14:57:39.892616 30076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260620 14:57:39.892757 30076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/wal
uuid: "a43617f1c5b644cdb07cc10969e5dae7"
format_stamp: "Formatted at 2026-06-20 14:57:39 on dist-test-slave-sms9"
server_key: "e5122f9d2eb3f9a79d7e0b8a23197bae"
server_key_iv: "5ad925ef9b972d085845581e9c8edfe1"
server_key_version: "encryptionkey@0"
I20260620 14:57:39.892889 30076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 14:57:39.903568 30076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 14:57:39.906348 30076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 14:57:39.906579 30076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 14:57:39.913475 30076 rpc_server.cc:307] RPC server started. Bound to: 127.7.24.252:46323
I20260620 14:57:39.913761 30144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.24.252:46323 every 8 connection(s)
I20260620 14:57:39.914561 30076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/data/info.pb
I20260620 14:57:39.915110 7267 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu as pid 30076
I20260620 14:57:39.915263 7267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGdDYa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781967176660325-7267-0/minicluster-data/master-2/wal/instance
I20260620 14:57:39.915570 7267 external_mini_cluster.cc:1428] Setting key cf3805b70499d38db75421a009335184
I20260620 14:57:39.917477 30145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 14:57:39.921134 30145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
I20260620 14:57:39.930088 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.916936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57578 (local address 127.7.24.252:46323)
0620 14:57:39.917332 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.917349 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.917382 (+ 33us) server_negotiation.cc:407] Connection header received
0620 14:57:39.918059 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.918075 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.918398 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.918775 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.919568 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.920547 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.921405 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.921936 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.922108 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.925094 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.925099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.925107 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.925133 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.926986 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.927495 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.927501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.927507 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.927646 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.927936 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.927939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.927941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.928392 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.928547 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.928905 (+ 358us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.929205 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":136,"threads_started":1}
I20260620 14:57:39.932753 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.922850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.7.24.254:38241)
0620 14:57:39.923013 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.923018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.923409 (+ 391us) server_negotiation.cc:407] Connection header received
0620 14:57:39.923568 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.923571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.923616 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.923678 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.924430 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.925159 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.926549 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.926757 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.926835 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.929319 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.929322 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.929324 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.929348 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.930909 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.931368 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.931374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.931378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.931459 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.931735 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.931740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.931741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.931979 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.932083 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.932511 (+ 428us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.932646 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 14:57:39.932798 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.922802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.254:38241 (local address 127.0.0.1:48288)
0620 14:57:39.923252 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.923312 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:39.923450 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:39.923765 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:39.923773 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:39.924173 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:39.924299 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.924304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.925274 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.925281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.926347 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.926358 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.926872 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.926877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.926995 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.927566 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:39.927579 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:39.929167 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:39.931024 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.931033 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.931039 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.931263 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.931554 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.931558 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.931560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.931644 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.932087 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:39.932110 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:39.932240 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 14:57:39.932436 (+ 196us) client_negotiation.cc:240] Negotiation successful
0620 14:57:39.932626 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"spinlock_wait_cycles":3968,"thread_start_us":165,"threads_started":1}
I20260620 14:57:39.934103 30145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.254@KRBTEST.COM
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
I20260620 14:57:39.941946 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.932413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.7.24.254:38241)
0620 14:57:39.932942 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.932948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.932963 (+ 15us) server_negotiation.cc:407] Connection header received
0620 14:57:39.933002 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.933005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.933050 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.933134 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.933871 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.934448 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.935578 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.935891 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.935995 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.938426 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.938431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.938438 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.938509 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.940297 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.940702 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.940706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.940708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.940793 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.941072 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.941077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.941078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.941263 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.941496 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.941627 (+ 131us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.941739 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":113,"threads_started":1}
I20260620 14:57:39.942377 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.934641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.253:36451 (local address 127.0.0.1:40652)
0620 14:57:39.934788 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.934801 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:39.934866 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:39.935092 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:39.935094 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:39.935270 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:39.935372 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.935378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.936396 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.936399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.936889 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.936896 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.937355 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.937358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.937457 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.937890 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:39.937901 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:39.939292 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:39.940885 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.940890 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.940893 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.941086 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.941375 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.941378 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.941380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.941464 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.941841 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:39.941844 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:39.941942 (+ 98us) client_negotiation.cc:769] Sending connection context
0620 14:57:39.942063 (+ 121us) client_negotiation.cc:240] Negotiation successful
0620 14:57:39.942204 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260620 14:57:39.942793 30074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.934757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40652 (local address 127.7.24.253:36451)
0620 14:57:39.934879 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.934884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.934915 (+ 31us) server_negotiation.cc:407] Connection header received
0620 14:57:39.934954 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.934956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.934995 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.935053 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.935487 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.936244 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.937013 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.937243 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.937351 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.939430 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.939434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.939437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.939469 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.940750 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.941186 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.941191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.941193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.941276 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.941560 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.941563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.941565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.941735 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.941851 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.942372 (+ 521us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.942527 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 14:57:39.943516 30145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.24.253@KRBTEST.COM
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.252@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
I20260620 14:57:39.951695 30074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.943446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40666 (local address 127.7.24.253:36451)
0620 14:57:39.943538 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.943544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.943560 (+ 16us) server_negotiation.cc:407] Connection header received
0620 14:57:39.943586 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.943588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.943649 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.943727 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.944404 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.944879 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.945642 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.945872 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.945982 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.948366 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.948373 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.948379 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.948407 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.949898 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.950373 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.950376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.950377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.950497 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.950786 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.950792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.950796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.951013 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.951227 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.951416 (+ 189us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.951523 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 14:57:39.951932 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.943992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57590)
0620 14:57:39.944157 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.944171 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:39.944247 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:39.944446 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:39.944448 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:39.944604 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:39.944659 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.944663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.945358 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.945364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.945862 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.945869 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.946966 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.946969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.947035 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.947508 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:39.947519 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:39.948612 (+ 1093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:39.950337 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.950342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.950344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.950629 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.950928 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.950930 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.950931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.951004 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.951407 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:39.951410 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:39.951506 (+ 96us) client_negotiation.cc:769] Sending connection context
0620 14:57:39.951613 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 14:57:39.951724 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260620 14:57:39.952390 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.944076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57590 (local address 127.7.24.252:46323)
0620 14:57:39.944158 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.944163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.944212 (+ 49us) server_negotiation.cc:407] Connection header received
0620 14:57:39.944313 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.944318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.944350 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.944439 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.944764 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.945253 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.945974 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.946825 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.946964 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.948722 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.948727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.948731 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.948760 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.950206 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.950727 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.950730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.950731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.950832 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.951097 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.951100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.951102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.951299 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.951393 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.952125 (+ 732us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.952256 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":36}
I20260620 14:57:39.953760 29994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } attempt: 3
I20260620 14:57:39.955619 30145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7: Bootstrap starting.
I20260620 14:57:39.956844 30069 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } attempt: 3
I20260620 14:57:39.958276 30145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7: Neither blocks nor log segments found. Creating new log.
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.254@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
I20260620 14:57:39.959208 30145 log.cc:824] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:39.961615 7267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 14:57:39.962618 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.954175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57594)
0620 14:57:39.954305 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.954322 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:39.954405 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:39.954740 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:39.954743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:39.954912 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:39.954997 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.955003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.955725 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.955729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.956247 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.956255 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.956726 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.956729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.956821 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.957298 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:39.957310 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:39.959008 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:39.960980 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.960984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.960986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.961252 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.961558 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.961561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.961564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.961664 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.962094 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:39.962096 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:39.962190 (+ 94us) client_negotiation.cc:769] Sending connection context
0620 14:57:39.962287 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 14:57:39.962440 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
Jun 20 14:57:39 dist-test-slave-sms9 krb5kdc[29913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781967459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.24.253@KRBTEST.COM for kudu/127.7.24.252@KRBTEST.COM
I20260620 14:57:39.963066 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.954363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57594 (local address 127.7.24.252:46323)
0620 14:57:39.954498 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.954504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.954517 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:57:39.954547 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.954550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.954592 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.954721 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.955111 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.955603 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.956367 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.956605 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.956725 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.959148 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.959151 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.959154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.959177 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.960869 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.961361 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.961364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.961366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.961475 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.961771 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.961774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.961776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.961965 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.962051 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.962823 (+ 772us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.962936 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 14:57:39.963156 30145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7: No bootstrap required, opened a new log
I20260620 14:57:39.963043 29968 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "af5b6769810a4c7ab902fe2d40ad8629"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48300
I20260620 14:57:39.966050 29994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629: Bootstrap starting.
I20260620 14:57:39.968298 29994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:39.969231 29994 log.cc:824] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:39.971325 29994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629: No bootstrap required, opened a new log
I20260620 14:57:39.971491 30145 raft_consensus.cc:348] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } }
I20260620 14:57:39.971755 30145 raft_consensus.cc:729] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43617f1c5b644cdb07cc10969e5dae7, State: Initialized, Role: FOLLOWER
I20260620 14:57:39.972057 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.957575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.24.252:46323 (local address 127.0.0.1:57598)
0620 14:57:39.957866 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0620 14:57:39.957881 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 14:57:39.957960 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 14:57:39.958299 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 14:57:39.958302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 14:57:39.958625 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0620 14:57:39.958727 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.958733 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.959702 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.959707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.960429 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 14:57:39.960440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.961027 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.961030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 14:57:39.961118 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.961613 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 14:57:39.961628 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 14:57:39.963244 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 14:57:39.970355 (+ 7111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.970359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.970361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.970677 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.971008 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 14:57:39.971011 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 14:57:39.971012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 14:57:39.971108 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 14:57:39.971566 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 14:57:39.971569 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 14:57:39.971669 (+ 100us) client_negotiation.cc:769] Sending connection context
0620 14:57:39.971755 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 14:57:39.971906 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260620 14:57:39.972101 30145 consensus_queue.cc:260] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } }
I20260620 14:57:39.972625 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.957624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57598 (local address 127.7.24.252:46323)
0620 14:57:39.957969 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.957981 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.957999 (+ 18us) server_negotiation.cc:407] Connection header received
0620 14:57:39.958038 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.958041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.958139 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.958239 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.958860 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.959551 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.960575 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.960900 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.961367 (+ 467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:39.963594 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:39.963598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:39.963600 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:39.963625 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:39.970212 (+ 6587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.970809 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.970812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.970814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.970898 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:39.971217 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:39.971220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:39.971221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:39.971444 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:39.972219 (+ 775us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:39.972315 (+ 96us) server_negotiation.cc:299] Negotiation successful
0620 14:57:39.972460 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":110,"threads_started":1}
I20260620 14:57:39.972965 30154 sys_catalog.cc:455] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } } }
I20260620 14:57:39.973275 30154 sys_catalog.cc:458] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:39.973440 30145 sys_catalog.cc:564] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:39.974658 29994 raft_consensus.cc:348] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } }
I20260620 14:57:39.974884 29994 raft_consensus.cc:729] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af5b6769810a4c7ab902fe2d40ad8629, State: Initialized, Role: FOLLOWER
I20260620 14:57:39.975248 29994 consensus_queue.cc:260] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } }
I20260620 14:57:39.975625 30069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44: Bootstrap starting.
I20260620 14:57:39.975941 30166 sys_catalog.cc:455] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } } }
I20260620 14:57:39.976074 30166 sys_catalog.cc:458] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:39.976804 29994 sys_catalog.cc:564] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 14:57:39.976868 29968 raft_consensus.cc:482] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 14:57:39.976956 29968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:39.977897 30069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44: Neither blocks nor log segments found. Creating new log.
I20260620 14:57:39.978305 29968 raft_consensus.cc:504] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } }
W20260620 14:57:39.979007 29947 tablet.cc:2369] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 14:57:39.979141 30069 log.cc:824] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44: Log is configured to *not* fsync() on all Append() calls
I20260620 14:57:39.979318 29968 leader_election.cc:288] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [CANDIDATE]: Term 1 election: Requested vote from peers 8aabf7cd1dc1442e8f0247ff9d451f44 (127.7.24.253:36451), a43617f1c5b644cdb07cc10969e5dae7 (127.7.24.252:46323)
I20260620 14:57:39.981987 30069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44: No bootstrap required, opened a new log
W20260620 14:57:39.982399 30177 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:57:39.982823 30177 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:57:39.982733 30119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af5b6769810a4c7ab902fe2d40ad8629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a43617f1c5b644cdb07cc10969e5dae7"
I20260620 14:57:39.982970 30119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:39.984149 30119 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af5b6769810a4c7ab902fe2d40ad8629 in term 1.
I20260620 14:57:39.984578 29945 leader_election.cc:302] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a43617f1c5b644cdb07cc10969e5dae7, af5b6769810a4c7ab902fe2d40ad8629; no voters:
I20260620 14:57:39.984867 30166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 14:57:39.985066 30166 raft_consensus.cc:686] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [term 1 LEADER]: Becoming Leader. State: Replica: af5b6769810a4c7ab902fe2d40ad8629, State: Running, Role: LEADER
I20260620 14:57:39.985394 30166 consensus_queue.cc:237] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } }
I20260620 14:57:39.986117 30069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } }
I20260620 14:57:39.986348 30069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aabf7cd1dc1442e8f0247ff9d451f44, State: Initialized, Role: FOLLOWER
I20260620 14:57:39.986697 30069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } }
I20260620 14:57:39.987187 30166 sys_catalog.cc:455] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af5b6769810a4c7ab902fe2d40ad8629. Latest consensus state: current_term: 1 leader_uuid: "af5b6769810a4c7ab902fe2d40ad8629" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } } }
I20260620 14:57:39.987293 30166 sys_catalog.cc:458] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:39.988103 30069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [sys.catalog]: configured and running, proceeding with master startup.
W20260620 14:57:39.988332 30165 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:57:39.988413 30165 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:57:39.989056 30043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af5b6769810a4c7ab902fe2d40ad8629" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44"
I20260620 14:57:39.989279 30043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [term 0 FOLLOWER]: Advancing to term 1
I20260620 14:57:39.990829 30043 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af5b6769810a4c7ab902fe2d40ad8629 in term 1.
I20260620 14:57:39.992265 30189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 14:57:39.993290 30189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 14:57:39.993032 29926 master_runner.cc:186] Error getting master registration for 127.7.24.253:36451: OK, instance_id { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" instance_seqno: 1781967459702044 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 14:57:39.996227 30187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } } }
I20260620 14:57:39.996374 30187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:39.996408 29926 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 14:57:39.992848 30076 master_runner.cc:186] Error getting master registration for 127.7.24.253:36451: OK, instance_id { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" instance_seqno: 1781967459702044 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 14:57:39.996883 30076 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 14:57:40.000800 30119 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [term 1 FOLLOWER]: Refusing update from remote peer af5b6769810a4c7ab902fe2d40ad8629: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:40.001780 30185 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:40.006170 30154 sys_catalog.cc:455] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af5b6769810a4c7ab902fe2d40ad8629. Latest consensus state: current_term: 1 leader_uuid: "af5b6769810a4c7ab902fe2d40ad8629" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } } }
I20260620 14:57:40.006318 30154 sys_catalog.cc:458] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:40.007532 30043 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [term 1 FOLLOWER]: Refusing update from remote peer af5b6769810a4c7ab902fe2d40ad8629: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 14:57:40.008119 30185 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 14:57:40.012526 30187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af5b6769810a4c7ab902fe2d40ad8629. Latest consensus state: current_term: 1 leader_uuid: "af5b6769810a4c7ab902fe2d40ad8629" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } } }
I20260620 14:57:40.012665 30187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:40.015676 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.991615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48314 (local address 127.7.24.254:38241)
0620 14:57:39.991724 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.991728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.006748 (+ 15020us) server_negotiation.cc:407] Connection header received
0620 14:57:40.006903 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.006906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.006956 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.007031 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.007880 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:40.008579 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:40.009667 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:40.009966 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:40.010051 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:40.011828 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:40.011835 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:40.011841 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:40.011872 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:40.013778 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:40.014259 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:40.014262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:40.014264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:40.014361 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:40.014692 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:40.014695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:40.014697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:40.014899 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:40.015002 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:40.015377 (+ 375us) server_negotiation.cc:299] Negotiation successful
0620 14:57:40.015501 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 14:57:40.018283 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.982795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57608 (local address 127.7.24.252:46323)
0620 14:57:39.990569 (+ 7774us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.990574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:39.990594 (+ 20us) server_negotiation.cc:407] Connection header received
0620 14:57:39.990643 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:39.990649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:39.990709 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:39.997374 (+ 6665us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:39.997502 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.998241 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:39.999481 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:39.999799 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:40.001919 (+ 2120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:40.002039 (+ 120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:40.002043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:40.002045 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:40.002063 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:40.003917 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:40.004443 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:40.004446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:40.004448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:40.004533 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:40.004854 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:40.004857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:40.004859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:40.005082 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:40.017819 (+ 12737us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:40.017921 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 14:57:40.018065 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7684}
I20260620 14:57:40.021222 30074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:39.990864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.7.24.253:36451)
0620 14:57:39.990981 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:39.990986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.005930 (+ 14944us) server_negotiation.cc:407] Connection header received
0620 14:57:40.006132 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.006136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.006195 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.006251 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.013210 (+ 6959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:40.013991 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:40.015090 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:40.015430 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:40.015504 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:40.017158 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 14:57:40.017162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 14:57:40.017165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 14:57:40.017195 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 14:57:40.018964 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:40.019659 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:40.019662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:40.019664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:40.019756 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 14:57:40.020046 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 14:57:40.020049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 14:57:40.020050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 14:57:40.020254 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 14:57:40.020382 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:40.020959 (+ 577us) server_negotiation.cc:299] Negotiation successful
0620 14:57:40.021072 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 14:57:40.023993 30185 sys_catalog.cc:455] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af5b6769810a4c7ab902fe2d40ad8629" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } } }
I20260620 14:57:40.024127 30185 sys_catalog.cc:458] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:40.026365 30187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af5b6769810a4c7ab902fe2d40ad8629" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } } }
I20260620 14:57:40.026546 30187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:40.026911 30166 sys_catalog.cc:455] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af5b6769810a4c7ab902fe2d40ad8629" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } } }
I20260620 14:57:40.027032 30166 sys_catalog.cc:458] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629 [sys.catalog]: This master's current role is: LEADER
I20260620 14:57:40.027918 30154 sys_catalog.cc:455] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af5b6769810a4c7ab902fe2d40ad8629" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af5b6769810a4c7ab902fe2d40ad8629" member_type: VOTER last_known_addr { host: "127.7.24.254" port: 38241 } } peers { permanent_uuid: "8aabf7cd1dc1442e8f0247ff9d451f44" member_type: VOTER last_known_addr { host: "127.7.24.253" port: 36451 } } peers { permanent_uuid: "a43617f1c5b644cdb07cc10969e5dae7" member_type: VOTER last_known_addr { host: "127.7.24.252" port: 46323 } } }
I20260620 14:57:40.028052 30154 sys_catalog.cc:458] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 14:57:40.028463 30203 mvcc.cc:205] Tried to move back new op lower bound from 7298938716155379712 to 7298938716103577600. Current Snapshot: MvccSnapshot[applied={T|T < 7298938716155379712}]
I20260620 14:57:40.029036 30189 catalog_manager.cc:1347] Generated new cluster ID: 874168cd85014a99bd2b3f3d5a95ad6d
I20260620 14:57:40.029098 30189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 14:57:40.033376 30202 mvcc.cc:205] Tried to move back new op lower bound from 7298938716155379712 to 7298938716103577600. Current Snapshot: MvccSnapshot[applied={T|T < 7298938716155379712 or (T in {7298938716155379712})}]
I20260620 14:57:40.035876 30205 mvcc.cc:205] Tried to move back new op lower bound from 7298938716155379712 to 7298938716103577600. Current Snapshot: MvccSnapshot[applied={T|T < 7298938716155379712}]
W20260620 14:57:40.044454 30207 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 14:57:40.044556 30207 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 14:57:40.075348 30189 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 14:57:40.076678 30189 catalog_manager.cc:1504] Loading token signing keys...
I20260620 14:57:40.092720 30189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af5b6769810a4c7ab902fe2d40ad8629: Generated new TSK 0
I20260620 14:57:40.093611 30189 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 14:57:40.127827 30151 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.124549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57622 (local address 127.7.24.252:46323)
0620 14:57:40.124706 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.124712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.126656 (+ 1944us) server_negotiation.cc:407] Connection header received
0620 14:57:40.126887 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.126891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.126960 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.127015 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.127664 (+ 649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57622: BlockingRecv error: recv got EOF from 127.0.0.1:57622 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260620 14:57:40.129885 30074 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.125291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.7.24.253:36451)
0620 14:57:40.125420 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.125424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.128219 (+ 2795us) server_negotiation.cc:407] Connection header received
0620 14:57:40.128813 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.128817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.128870 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.128920 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.129739 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40682: BlockingRecv error: recv got EOF from 127.0.0.1:40682 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260620 14:57:40.130673 29998 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.126007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48322 (local address 127.7.24.254:38241)
0620 14:57:40.126126 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.126130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.128542 (+ 2412us) server_negotiation.cc:407] Connection header received
0620 14:57:40.129059 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.129062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.129114 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.129164 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.130551 (+ 1387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48322: BlockingRecv error: recv got EOF from 127.0.0.1:48322 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260620 14:57:40.135638 30074 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.133974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40696 (local address 127.7.24.253:36451)
0620 14:57:40.134083 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.134087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.134561 (+ 474us) server_negotiation.cc:407] Connection header received
0620 14:57:40.134787 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.134791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.134866 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.134938 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.135520 (+ 582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40696: BlockingRecv error: recv got EOF from 127.0.0.1:40696 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260620 14:57:40.140911 30074 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.139353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40710 (local address 127.7.24.253:36451)
0620 14:57:40.139459 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.139463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.140003 (+ 540us) server_negotiation.cc:407] Connection header received
0620 14:57:40.140188 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.140192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.140247 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.140297 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.140805 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40710: BlockingRecv error: recv got EOF from 127.0.0.1:40710 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 14:57:40.146700 30074 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.145025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40718 (local address 127.7.24.253:36451)
0620 14:57:40.145139 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.145143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.145680 (+ 537us) server_negotiation.cc:407] Connection header received
0620 14:57:40.145884 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.145888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.145941 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.145997 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.146572 (+ 575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40718: BlockingRecv error: recv got EOF from 127.0.0.1:40718 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 14:57:40.154901 30074 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.153047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.7.24.253:36451)
0620 14:57:40.153224 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.153231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.153844 (+ 613us) server_negotiation.cc:407] Connection header received
0620 14:57:40.154060 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.154065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.154123 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.154180 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.154771 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40732: BlockingRecv error: recv got EOF from 127.0.0.1:40732 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260620 14:57:40.167524 30074 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.165609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40746 (local address 127.7.24.253:36451)
0620 14:57:40.165742 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.165747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.166400 (+ 653us) server_negotiation.cc:407] Connection header received
0620 14:57:40.166665 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.166668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.166729 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.166796 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.167398 (+ 602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40746: BlockingRecv error: recv got EOF from 127.0.0.1:40746 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260620 14:57:40.188385 30074 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.186342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.7.24.253:36451)
0620 14:57:40.186507 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.186512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.187239 (+ 727us) server_negotiation.cc:407] Connection header received
0620 14:57:40.187511 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.187517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.187592 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.187663 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.188268 (+ 605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40758: BlockingRecv error: recv got EOF from 127.0.0.1:40758 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260620 14:57:40.224581 30074 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.223257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40770 (local address 127.7.24.253:36451)
0620 14:57:40.223392 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.223397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.223658 (+ 261us) server_negotiation.cc:407] Connection header received
0620 14:57:40.223834 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.223837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.223897 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.223990 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.224461 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40770: BlockingRecv error: recv got EOF from 127.0.0.1:40770 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260620 14:57:40.292011 30074 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.290730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40786 (local address 127.7.24.253:36451)
0620 14:57:40.290938 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.290942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.291096 (+ 154us) server_negotiation.cc:407] Connection header received
0620 14:57:40.291265 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.291269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.291335 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.291430 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.291887 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40786: BlockingRecv error: recv got EOF from 127.0.0.1:40786 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260620 14:57:40.423739 30074 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.422411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40796 (local address 127.7.24.253:36451)
0620 14:57:40.422580 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.422585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.422741 (+ 156us) server_negotiation.cc:407] Connection header received
0620 14:57:40.422950 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.422954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.423002 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.423101 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.423596 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40796: BlockingRecv error: recv got EOF from 127.0.0.1:40796 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260620 14:57:40.683146 30074 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 14:57:40.681869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.7.24.253:36451)
0620 14:57:40.682032 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:40.682037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:40.682157 (+ 120us) server_negotiation.cc:407] Connection header received
0620 14:57:40.682338 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:40.682341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:40.682390 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:40.682510 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0620 14:57:40.683011 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40810: BlockingRecv error: recv got EOF from 127.0.0.1:40810 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 14:57:40.989894 30165 catalog_manager.cc:1259] Loaded cluster ID: 874168cd85014a99bd2b3f3d5a95ad6d
I20260620 14:57:40.990015 30165 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7: loading cluster ID for follower catalog manager: success
I20260620 14:57:40.991734 30165 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7: acquiring CA information for follower catalog manager: success
I20260620 14:57:40.992915 30165 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a43617f1c5b644cdb07cc10969e5dae7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 14:57:41.046106 30207 catalog_manager.cc:1259] Loaded cluster ID: 874168cd85014a99bd2b3f3d5a95ad6d
I20260620 14:57:41.046217 30207 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44: loading cluster ID for follower catalog manager: success
I20260620 14:57:41.047973 30207 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44: acquiring CA information for follower catalog manager: success
I20260620 14:57:41.049157 30207 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8aabf7cd1dc1442e8f0247ff9d451f44: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 14:57:41.201610 30277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:41.197452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.7.24.253:36451)
0620 14:57:41.197705 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:41.197710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:41.197820 (+ 110us) server_negotiation.cc:407] Connection header received
0620 14:57:41.198012 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:41.198016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:41.198147 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:41.198273 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:41.198564 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:41.199108 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:41.200058 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:41.200292 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:41.200406 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:41.200742 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:41.201192 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:41.201337 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:41.201377 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 14:57:41.201440 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":67,"threads_started":1}
I20260620 14:57:41.208359 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:41.204820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.7.24.252:46323)
0620 14:57:41.205045 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:41.205049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:41.205062 (+ 13us) server_negotiation.cc:407] Connection header received
0620 14:57:41.205220 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:41.205222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:41.205338 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:41.205467 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:41.205652 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:41.206145 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:41.207024 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:41.207246 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:41.207325 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:41.207605 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:41.208016 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:41.208104 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:41.208185 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 14:57:41.208235 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
I20260620 14:57:41.216068 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:41.211697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57636 (local address 127.7.24.252:46323)
0620 14:57:41.211826 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:41.211830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:41.212114 (+ 284us) server_negotiation.cc:407] Connection header received
0620 14:57:41.212342 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:41.212345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:41.212392 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:41.212468 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:41.212763 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:41.213641 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:41.214867 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:41.215112 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:41.215258 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:41.215553 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:41.215644 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:41.215718 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:41.215848 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 14:57:41.215916 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 14:57:41.216459 30277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:41.211623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40826 (local address 127.7.24.253:36451)
0620 14:57:41.211829 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:41.211835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:41.212088 (+ 253us) server_negotiation.cc:407] Connection header received
0620 14:57:41.212403 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:41.212406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:41.212485 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:41.212554 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:41.212983 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:41.213846 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:41.215049 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:41.215427 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:41.215508 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:41.215931 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:41.216010 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:41.216054 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:41.216233 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 14:57:41.216314 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260620 14:57:41.216882 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 14:57:41.211935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48334 (local address 127.7.24.254:38241)
0620 14:57:41.212196 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0620 14:57:41.212201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 14:57:41.212220 (+ 19us) server_negotiation.cc:407] Connection header received
0620 14:57:41.212325 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 14:57:41.212329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 14:57:41.212500 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 14:57:41.212619 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 14:57:41.212895 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:41.213590 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:41.214921 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 14:57:41.215286 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 14:57:41.215391 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 14:57:41.215738 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 14:57:41.216106 (+ 368us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 14:57:41.216516 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0620 14:57:41.216640 (+ 124us) server_negotiation.cc:299] Negotiation successful
0620 14:57:41.216710 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":71,"threads_started":1}
I20260620 14:57:41.220683 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 29926
I20260620 14:57:41.227638 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 30001
I20260620 14:57:41.233668 7267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGdDYa/build/debug/bin/kudu with pid 30076
2026-06-20T14:57:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3124 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6234 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (284567 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST